Streamline your List of Documentaries Celebrating the First Anniversary of the Syrian Revolution with the Command Line YT, OpenAI, & NLP Cloud

A lot of different Arabic speaking media throughout the Arabic speaking world before or right before the first anniversary of Syria’s revolution published documentaries. Many of these documentaries are raw; they have no subtitles, Arabic or English; they are not summarized; they have do not have logically categorized chapter pinned to a specific time in the documentary in an outline; they are not labeled, differentiated, specified for length, speakers, or analysis. They are published without any additional information.

You will coordinate the use of four tools in a script for the Command Line Interface (henceforth CLI). You will streamline processing for your list of documentaries with the script. The first tool is called YouTube Downloader. It is an extremely robust open source program available on GitHub for downloading publicly available material from streaming websites. The second tool is OpenAI’s Whisper. OpenAI’s Whisper is a CLI tool for generating transcripts from publicly available material like the ones downloaded from streaming sites. It is a major ingenuity for human kind; OpenAI’s Whisper provides developers with the ability to transcribe transcripts from a Desktop computer. The third tool is NLP Studio’s API for summarization. NLP Studio’s API for summarization stages a number of the most widely used summarization Large Language Models (henceforth LLMs) such as GPT-OSS 120B, LLaMA 3.1 405B, Fine-tuned LLaMA 3.3 70B, ChatDolphin, Dolphin Yi 34B, Dolphin Mixtral 8x7B, Bart Large CNN, T5 Base EN Generate Headline; these LLMs empower users with the ability to chose what might be best. You will bring everything together from video codecs to transcripts, summarizations to clips, with the ffmpeg tool, the fourth tool. You will collect these tools into a single script to streamline the implementation of these tools for processing.

While there are many documentaries celebrating Syria’s First Anniversary of its successful military coup d’état, you will limit your focus to no more than five of the most prominent. These are in successive order: 1) مفاتيح دمشق (i.e., Al-Jazeera), 2) سقوط منتظر .. كواليس عملية ردع العدوان وخفايا انهيار جيش بشار الأسد (i.e., Al-Araby), 3) خفايا تُكشف للمرة الأولى في معركة ردع العدوان | فيلم ساعة الصفر (i.e., Syria-TV), and 4) رحيل بلا عودة.. وثائقي يكشف اللحظات الحاسمة التي مهدت لسقوط نظام بشار الأسد وقصة هروبه المفاجئة (i.e., Al-Arabiya); 5) هل سقط الأسد بـ “قرار دولي”؟ خفايا الانهيار السريع وقصة ليلة تحرير دمشق | وثائقي (i.e., Almodon Plus).

These documentaries are exclusively from the Arab world; since they are in Arabic without English translations, they are largely inaccessible to the vast majority of the world’s population. AI, however, opens up the world. With the ability to translate documentaries like these about Syria’s first anniversary of its revolution, AI literally opens the door to the Arab World.

It is developers, however, who hold the keys to these doors. Unlocking the power of technology with key knowledge, developers, who have spent hour after hour concentrating on the power of the terminal, commands or, scripts, hold the keys to technology like OpenAI for unlocking the Arabic language.

Step 1

Your first step is to head over to GitHub to download the YouTube downloader tool. The link is the following: https://github.com/yt-dlp. Since YT-DLP provides a lot of different options for installing the program, you can chose: https://github.com/yt-dlp/yt-dlp/wiki/Installation.

To download one or more these publicly available documentaries, you run the following command:

yt-dlp "<link>"

The <link> entry is the full link. It could be hosted on a number of different sites but typically the full link is everything before the first &, there are a lot of things on the link. If not, the first line after a = may be the full link up or until the first &.

Step 2

Your second step is to download OpenAI’s Whisper. The link is the following: https://github.com/openai/whisper. OpenAI’s whisper is available for multiple different operating systems so its instructions on installation covers the most common. https://github.com/openai/whisper?tab=readme-ov-file#setup

To use Whipser in the CLI, you run only a few commands. The first is the command to transcribe. Since these documentaries are in Arabic, you can transcribe them with the following command:

whisper documentary.mp4 --model medium --language Arabic --output_format srt

With your .srt you have what you need to produce a summary.

Step 3

Your second step is to create an account at NLP Cloud. After you create an account, grab the API key so that you can authenticate. Since you’re focused on the CLI, you will use curl for your API requests. You will collect sections of the transcripts into chunks. You will use these chunks for submitting your POST requests to NLP Cloud.

Create a curl request. Take your API key. Where <token> stands, add your API key. The following is an example:

curl "https://api.nlpcloud.io/v1/<model_name>/summarization" \
  -H "Authorization: Token <token>" \
  -H "Content-Type: application/json" \
  -X POST -d '{"text":"This army was hiding behind Iranian support, 
which proved to be a failure The Iranians will continue in this operation
 after the Lebanon operation Also the Russians were heavily involved in the Ukraine 
war so they could not be as effective as in 2015."}'

The return data is a summary of your chunk.

Step 4

Your fourth step is to download the open source software at GitHub for ffmpeg at the following website: https://github.com/FFmpeg/FFmpeg. With ffmpeg everything comes together. You run a series of commands on the videos. The first command converts the .srt into a .ass file.

ffmpeg -i transcript_of_documentary.srt -c:s ass transcript_of_documentary.ass

The .ass format is better for the Arabic language, since the .srt format often contains inexplicable squares in the font. With the transcript’s format converted, the next step is to add the transcripts.

ffmpeg -i documentary.mp4 -vf ass=transcript_of_documentary.ass documentary_output_subtitles.mp4

With the converted transcripts added to the video, you can now add the summarized chunks to the metadata of your documentary.

The Script

Your last step is to create a bash shell script. The bash shell script is composed of nested if statements. the user to provide a link

#!/bin/bash

# Ask the user if they want to download a video
read -p "Do you want to download a video? (yes/no): " RESPONSE

# Check if user's response is yes 
if [[ "$RESPONSE" == "yes" ]]; then 
# since user's response is yes, execute the following: 

    # Prompt the user for link
    echo -e "\033[31mGet your link ready\033[0m"
    read -p "Enter the link: " VIDEO_URL
    echo "Your link is: $VIDEO_URL!"
    FILENAME=$(yt-dlp --get-filename -o 'video.%(ext)s' "$VIDEO_URL")
    echo "The video will be saved as: $FILENAME"
    echo "We're downloading the file now."
    yt-dlp -o "$FILENAME" "$VIDEO_URL"
        if [$? -eq 0]; then # code
            echo "The video downloaded successfully."
        else 
            echo "The video did not download successfully." 
        fi
else 
    echo "The script is terminating." 
    pkill -9 ./prompt.sh

fi

# Ask the user if they want to download a video 
read -p "Do you want to transcribe your video? (yes/no): " responseToProcessing

# Check if user's response is yes
if [[ "$responseToProcessing" == "yes" ]]; then 
# since user's response is yes, execute the following: 

    # Prompt the user for
    echo -e "\e[31m Get your link ready. \e[0m"
    read -p "Enter the language: " LANGUAGE
    echo "Your language is: $LANGUAGE!"
    echo "We're starting the transcription now..."
    whisper $FILENAME --model medium --language "$LANGUAGE" --output_format srt
        
        # Declare variables for handling .srt/.ass
        FULLFILENAME=$(basename -- "$FILENAME")
        EXTENSION="${FILENAME##*.}"
        FILENAMEWOEXT="${FILENAME%.*}"

        echo "Here is the file name $FILENAME"
        echo "Here is the $EXTENSION" 
        echo "Here is the file name without the extension: $FILENAMEWOEXT"
        
        # Check if whisper executed correctly
        if [ $? -eq 0 ]; then
            FILENAMEWOEXT="${FILENAME%.*}"
            echo "$FILENAMEWOEXT.srt exists!"
        else
            echo "FFmpeg command failed."
        fi
else 
    echo "The script is terminating." 
    pkill -9 prompt
fi

# Check if the .srt file exists
if [[ -e "$FILENAMEWOEXT.srt" ]]; then 

# since the .srt exists, execute the following: 
    echo "You have an $FILENAMEWOEXT.srt now!" 
    echo "We're converting your .srt to an .ass"
    ffmpeg -i $FILENAMEWOEXT.srt -c:s ass $FILENAMEWOEXT.ass
        if [ $? -eq 0 ]; then
            echo "FFmpeg command succeeded!"
        else
            echo "FFmpeg command failed."
        fi
else 
    echo "There is no .srt yet!" 
fi 

# Check if the .ass file exists
if [[ -e "$FILENAMEWOEXT.ass" ]]; then # code to execute if true; 

    # since the .ass exists, execute the following: 
    # delete the .srt
    rm -rf $FILENAMEWOEXT.srt

    # append the .ass to the video
    echo "We're appending your transcripts to your .mp4"
    ffmpeg -i "$FILENAME" -vf ass="$FILENAMEWOEXT.ass" "$FILENAME"+"_output_subtitles".mp4
        if [ $? -eq 0 ]; then
            echo "FFmpeg command succeeded with appending transcripts to your $EXTENSION!"
        else
            echo "FFmpeg command failed."
        fi

else 
    echo "There's no .srt" 
fi

exit 1

Conclusion

With the commands from these four different tools, you can create a script, bringing the four of them together with prompts. The script prompts the user for a link. The downloaded video is passed over to WhisperAI, which processes the video for transcripts. You add the return data from NLP Cloud’s responses to the chunks of the transcripts into a file. The transcripts are converted from an .srt file to an .ass file before being added by the same command to the video itself.

Disadvantages

There are a few disadvantages. In a separate iteration with a different scripting language like Python, a more robust user prompt with far more safeguards against an unfriendly user experience (i.e., such as a message about Internet connectivity), far more layered `.json` processing for granularized inserts of the parsed response data from the NLPCloud.io API endpoint, or the implementation of programming principles such as the separation of concerns with individual functions (i.e., replacing the nested if statements), would all be a desirable.

Advantages

Despite the disadvantages, bash, nonetheless, produces results. With the material fully processed, you can create a schedule with crontabs that enables you to watch automatically the processed videos during your lunch hour so that you can make December 8th last five days instead of only one. Imagine being able to enjoy the last revolution in the Arab Spring during lunch for a whole week without having to write a program. It is just a script!

Endnote

For the meantime, the minimalist design of the script provides what is need to limit the number of repetitive commands an avid streaming programmer may have to enter to schedule his or her own personal TV Guide with fully transcribed Arabic documentaries on the fall of Bashar Al-Assad.

AI cannot come close to the preferences — always motivated by an intuitive, innate, heartfelt sense of rightful righteousness like Socrates own mysterious εὐδαιμονία — producing that streaking red line of thought called passion. It just cannot. It is hardly a summary, since there is no hard logic to its decision to include some but preclude other facts. A summary is always a personal reflection, an essai, on one’s experience of an event. The summary APIs are the least powerful, least promising, least interesting algorithms.

A summary is an opinion, plain and simple, and no machine, no matter how fantastic, is capable of producing an opinion, and so AI generated summaries, the vast majority of which inherit a human being’s opinion from the leading piece of data (i.e., the centerpiece) will lack the human touch so important for what a summary truly is. AI’s inability to generate music, especially classical music, demonstrates its limitations; it is merely an aggregator, whose production of aggregations never exceeds the threshold required for a qualitative transformation of quantified data. There is no way for AI to imitate, mock, or emulate the human mind for lack of experience with real life and its affect in producing the attachments serving a guideposts in any expression, let alone summary.

The APIs focusing on the sentence, phase, or word are far more useful; far more interesting, far more promising for in these algorithms the manifold to a new universe of connections opens up. It is not one resulting from the reduction but the transformation of information, an interpretation that is neither an infringement nor an imposition but a predictable extension of a theory.

Summaries are not useful, since the AI algorithms for their production lack a human touch for something undeniably personal. An outline API endpoint must come to replace summaries. An algorithm capable of parsing information into sections, subsections, or sub-subsections for reference, navigation within a document, book, program or film, or recall are far more useful, especially since the amount of information a person consumes at 200 to 300 wpms far exceeds the average ability to handle more than a small percentage of the wpms for any given minute (even though human beings do not remember words but images).

Post Graduate Certification

Tangential programs like the one offered at the University of Texas at Austin called a “Post Graduate Program in Artificial Intelligence & Machine Learning: Business Applications” typifies the type of course universities are seeking to exploit as a side hustle for graduate programs. Offered by the McCombs School of Business, the program is slated to take no less than 7 months. It includes the study of Python for AI, Ensemble Techniques & Model Tuning, Deep Learning (Neural Networks), Natural Language Processing, Supervised and Unsupervised Learning, Recommendation Systems, and Computer Vision. Delivered by the learning platform ‘Great Learning,’ taught by professors from a university, the course is one of the many platforms forbuilding a strong foundation in machine learning.

Codeacademy

Other platforms are more focused on programming than the concepts like Codeacademy. Codeacademy offers a program geared directly towards the profession a learner assumes upon completion: Machine Learning / AI Engineer. The syllabus is really straightforward. The syllabus covers 7 units, 39 lessons, 37 projects, 33 quizzes. The first module is called “Introduction to Machine Learning Engineering Career Path,” the second, “Machine Learning Fundamentals,” the third “Software Engineering for Machine Learning/AI Engineers,” the fourth “Intermediate Machine Learning,” the fifth, “Building Machine Learning Pipelines,” the sixth “Machine Learning/AI Engineer: Final Portfolio,” the seventh, “Machine Learning Career Paths: Next Steps.” The course offers a certificate of completion. Thematically centered on Netflix, the course asks: “Ever wonder how Netflix recommends what you should watch?”

DataCamp

Codeacademy’s is slightly different from the ones DataCamp offers. DataCamp offers a series of different ‘career tracks’ based on a large body of Python courses. The two ‘career tracks’ DataCamp offers are 1) “Associate AI Engineer for Data Scientists” and 2) “Associate AI Engineer for Developers.” These are two extremely different career tracks. The first is focused on the following courses: a) “supervised learning with scikit-learning,” b) “unsupervised learning in Python,” c) “introduction to deep learning with Pytorch,” d) “developing multi input models for OCR,” e) “responsible for AI data management,” f) “introduction to LLMs in Python,” g) “analyzing car reviews with LLMs,” h) “working with Llama 3,” i) “MLOps Concepts,” j) “MLOps Deployment and LifeCycling,” k) “Introduction to Git,” l) “software engineering principles in Python”, m) “introduction to testing in Python”, n) “monitoring machine learning concepts.” This track appears to differ from the developer track in that this track focuses less on the actual models than on the science, theory, or concepts behind LLMs with a strong emphasis on programming in Python. The second career track appears to be focused on the actual models like the blockbuster APIs. The first course is a) “Working with OpenAI API,” b) “ChatGPT Prompt Engineering for Developers,” c) “Working with HuggingFace,” d) “LLMOps Concepts,” e) “Developing AI Systems with the OpenAI API,” f) “Organizing Medical Transcriptions with the OpenAI API,” g) “Introduction to Embeddings with the OpenAI API,” h) “Topic Analysis with Clothing Reviews with Embeddings,” i) “Vector Databases for Embeddings Pinecone,” j) “Software Engineering Principles in Python,” k) “Developing LLM Applications with LangChain.” The latter career track is clearly focused on implementation.

Deep Atlas

Last but not least is a new program called DeepAtlas. The program is unique from the post-graduate certificate, online certifications, or career tracks. In line with the self-guided style of these courses, the program takes place biweekly or monthly with bootcamp-style cohort. The bootcamp is thus either two weeks long or four depending on the time commitment the student is prepared to make.

The program is a community driven educational experience. The Deep Atlas community is comprised of “a select group of talented technologist who want to build a strong foundation in ML.” The curriculum is divided into four parts: ‘fundamentals,’ ‘shallow learning,’ ‘deep learning,’ and ‘leveraged ML.’ In the ‘fundamentals’ students learn Problem Framing, Environment Setup, Linear Algebra Fundamentals, Data Tools and Techniques, Simple Linear Regression, Building with LLMs, Probability and Statistics, Intro to Neural Networks, and the ML Terrain. In the ‘shallow learning,’ students learn Spatializing Data, Vector & Semantic Search, Inference & Training Mechanics, Model Development Lifecycle, Multiple Linear Regression, Gradient Boosted Trees, Taxonomy, of Shallow Algorithms, Dimensionality Reduction, Classification & Clustering, Nondeterministic Paradigm Shift, and Anomaly Detection. As for the other two parts, the Deep Atlas syllabus goes into more detail, if you seeking a blow-by-blow of the curriculum, and for that you have to sign up here.

Out of all of the pathways for learning ML, Deep Atlas’ program is the shortest with the most intense focus on the fundamentals, as opposed to the other platforms. While many people prefer seven-month long courses or an emphasis on transient tools and libraries, the Deep Atlas program focuses exclusively on enrolling seasoned software engineers who want to compress their studies with a focus on the fundamentals. Sign up here!

References

Microsoft Reactor Community : https://developer.microsoft.com/en-us/reactor/

The hardest of Python for MacOS users isn’t the language. It is the installation. Python’s numerous versions together with Mac’s lack of a package manager (just like Microsoft before the company introduced Microsoft Package Manager) make Python much more difficult than the language itself.

The easiest way to get and running with Python is to install Brew. Brew is the package manager Mac should have but doesn’t. To install Brew, run the following command:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

After Brew is installed, run the following command:

python --version

If you find no version, the next step is clear.

command not found: python

You install Python with Brew.

brew install python

If installed successfully, you will run into various ‘unversioned symlinks.’ The ‘unversioned symlinks’ are the most important missing part, since without the linkage for the following files, Python won’t run.

Python has been installed as
  /opt/homebrew/bin/python3

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /opt/homebrew/opt/python@3.12/libexec/bin

See: https://docs.brew.sh/Homebrew-and-Python

There is the problem! To fix the problem, you have to make your latest version of python the default version. To do so, access your `~/.bash_profile`.

nano ~/.bash_profile

The next step is to add Python to your $PATH. MacOS relies upon the Unix kernel but unlike more Linux distributions, MacOS does not provide the full functionality of a Linux distribution. The command wget, for instance, is one of the most commonly known aspects of the full functionality of a Linux distribution but no MacOS provides the command. The command together with the $PATH to its directory for executing the executable file for the command’s script must be installed just like Python, except Python is not a part of the full functionlaity of a Linux distribution like wget.

export PATH="/opt/homebrew/opt/python@3.11/libexec/bin:$PATH"

With the hardest part of the problem of Python out of the way, the fun begins. Python, which is a lot easier to operate than install on MacOS, offers more than merely a language. There is also a lot of libraries. One of these libraries is called requests. The library is a great way to connect to your computer to data on the Internet. If, for instance, you would like to download an article with something other than a wget script, you can use requests.

Downloading the Article

The first step is to create a data.py file in your editor. After creating the file, add a line to import the packages.

# Import packages
import requests

With the library accessible to your file, the next step is to save the url to your HREF as a variable.

# Specify url
url = ''

With the url saved as url, you have to set up your request.

# Package the request, send the request and catch the response: r
r = requests.get(url)

After fetching the url data with the .get request, the best thing to do is to return the data with a print().

# Package the request, send the request and catch the response: r
r = requests.get(url)

To print the data immediately, execute the .text method within a print() statement.

# printing request text
print(r.text)

The result is really unstructured data like soup that you just might like to beautify.

Printing the Soup Beautifully

Rename your variable to html_doc, executing the .text method on the data

# Extracts the response as html: html_doc
html_doc = r.text

To print the soup beautifully, you will use the Python library called BeautifulSoup, which is designed to parse HTML documents.

Up at the top of your document, add the following lines.

# Import packages
from bs4 import BeautifulSoup

With the BeautifulSoup library added to the document, create your frist instance of the object, just like below.

# create a BeautifulSoup object from the HTML: soup
soup = BeautifulSoup(html_doc, "html.parser" )

# With the BeautifulSoup library you can easily access different parts of a webpage with its "html.parser" argument, which parses the website for free. Isn’t the world of APIs a huge relief?

#Print the webpage's title 
print(soup.title)

Collecting the HREF

With the parser at hand, the way to scrape the HREF links from an article is really simple.

# Find all 'a' tags (with which HTML tags for HREFs begin): a_tags
a_tags = soup.find_all('a')

Scraping the HREFs requires no more than a single for loop.

# Print the URLs to the shell
for link in a_tags:
  print(link.get('href'))

Scraping the HREFs requires no more than a single for loop. After adding the for loop, run your script.

python data.py

Since you linked your command for python in your $PATH to your latest version of Python, you do not have to do anything more than hit enter.

The result should be a list of HREFs.

# Import packages
import requests
from bs4 import BeautifulSoup

# Specify url
url = 'https://uk.news.yahoo.com/russian-commander-fighting-ukraine-shot-125650646.html'

# Package the request, send the request and catch the response: r
r = requests.get(url)

# Extracts the response as html: html_doc
html_doc = r.text

# create a BeautifulSoup object from the HTML: soup
soup = BeautifulSoup(html_doc, "html.parser" )

# Print the title of Guido's webpage
print(soup.title)

# Find all 'a' tags (which define hyperlinks): a_tags
a_tags = soup.find_all('a')

# Print the URLs to the shell
for link in a_tags:
    print(link.get('href'))

The results are the following:

https://www.standard.co.uk/topic/russian
https://www.standard.co.uk/topic/ukraine
https://www.standard.co.uk/topic/telegram
https://uk.news.yahoo.com/ukraine-making-point-treating-russian-115822776.html
https://uk.news.yahoo.com/ukraine-strikes-ferry-crossing-occupied-084729225.html
https://uk.news.yahoo.com/russian-state-tv-makes-rare-135623498.html
https://uk.news.yahoo.com/even-kremlin-paid-liars-begging-144957546.html
https://uk.news.yahoo.com/woman-enjoying-quiet-dip-middle-135944280.html
https://uk.news.yahoo.com/deeply-unhinged-getting-worse-trump-084220362.html
https://uk.news.yahoo.com/oops-trump-accidentally-hands-kamala-033027506.html
https://uk.news.yahoo.com/putin-promised-poorly-trained-conscripts-040502408.html
https://uk.news.yahoo.com/soon-jobless-biden-unloads-trump-201336550.html
https://uk.news.yahoo.com/humiliation-putin-over-100-troops-131050434.html
https://uk.news.yahoo.com/revealed-ukraine-plan-force-putin-162310337.html
https://uk.news.yahoo.com/ukraine-russia-war-live-joint-033914158.html
https://uk.news.yahoo.com/trumpworld-scrambling-clean-mess-ex-172121250.html
https://uk.news.yahoo.com/biden-hits-trump-nickname-slams-222419224.html
https://uk.news.yahoo.com/kamala-harris-beat-donald-trump-180442262.html
https://uk.news.yahoo.com/vladimir-putin-ally-alexander-lukashenko-022100956.html
https://uk.news.yahoo.com/ukraine-captures-town-sudzha-russia-143139664.html
https://uk.news.yahoo.com/seth-meyers-stunned-trumps-olympic-100906141.html
https://uk.finance.yahoo.com/news/russia-doubling-pay-recruit-more-224312977.html
https://uk.news.yahoo.com/cnn-fact-checker-exposes-2-124212445.html
https://uk.news.yahoo.com/
https://uk.news.yahoo.com/
https://uk.news.yahoo.com/tagged/ukoriginal/
https://uk.news.yahoo.com/uk/
https://uk.news.yahoo.com/weather/
https://uk.news.yahoo.com/tv-celebrity/
https://uk.yahoo.com/topics/yahoo-quizzes-uk/
https://uk.yahoo.com/topics/crime/
https://uk.news.yahoo.com/science-tech/
https://uk.yahoo.com/topics/royal-family-news/
https://uk.news.yahoo.com/motoring/
https://uk.news.yahoo.com/viral/
https://uk.news.yahoo.com/world/
https://uk.yahoo.com/topics/on-this-day/
https://uk.news.yahoo.com/video/
https://uk.finance.yahoo.com/
https://uk.sports.yahoo.com/
https://uk.style.yahoo.com/
https://uk.movies.yahoo.com/
https://guce.yahoo.com/terms?locale=en-GB
https://guce.yahoo.com/privacy-policy?locale=en-GB
https://guce.yahoo.com/privacy-dashboard?locale=en-GB
https://uk.help.yahoo.com/kb/news
https://yahoo.uservoice.com/forums/168902-uk-news
https://www.adtech.yahooinc.com/en-gb
https://legal.yahoo.com/ie/en/yahoo/aboutourads/index.html
https://tumblr.com/follow/yahoonews-uk
https://www.twitter.com/YahooNewsUK
https://www.facebook.com/YahooUK
https://app.appsflyer.com/id304158842?pid=inproduct&c=US_Acquisition_YMktg_Desktopfooter__&af_sub1=Acquisition&af_sub2=US_YMktg&af_sub3=&af_sub4=&af_sub5=Desktopfooter__Static_
https://app.appsflyer.com/com.yahoo.mobile.client.android.yahoo?pid=inproduct&c=US_Acquisition_YMktg_YNewsfooter1__&af_sub1=Acquisition&af_sub2=US_YMktg&af_sub3=&af_sub4=&af_sub5=YNewsfooter1__Static_

Symbl.ai’s conversation AI platform provides developers with the ability to visualize, transform or connect a conversation far beyond merely live captioning transcripts in real-time.

Symbl.ai’s ability to detect contextual AI insights in real-time empowers developers to connect the results of their real-time conversations to Third Party SaaS providers such as JIRA. With the ability to log questions, follow-ups, or action-items in real time, Symbl.ai’s conversation AI platform empowers developers to process conversation data creating automatically. It easily integrates with Paragon, which helps with building productized integrations. Paragon’s embedded integration platform, for instance, is for product developers and in this case voice product developers that want to provide user-facing integrations between their app and their users’ other apps just like the one we’ve envisioned here for connecting Symbl.ai with JIRA. Enabled on the Paragon embedded integrations platform, Symbl.ai’s technology enables speakers in a conversation to process their conversation’s action-items automatically to JIRA. No post-meeting summaries or note taking is required. The JIRA tickets from action-items detected with Symbl.ai’s conversation AI platform update in a JIRA board automatically.

In the following blog you add Symbl.ai’s Action Items to JIRA in Real-Time with the Paragon Connect API. You integrate Symbl.ai’s real-time action items over a WebSocket’s connection, establishing the real-time tracking of conversational AI insights such as follow-ups, questions, or action items. After logging real-time conversational AI insights, you configure your WebSocket’s ws.onmessage method to capture action items specifically with a method called detectInsight. A previous blog helps developers onboard with logging action_items here. With action-items captured you will create a connection to Paragon’s embedded native SaaS API, making a call through its Connect API to deliver the Symbl.ai’s action item payload to a JIRA ticket.

JIRA

JIRA is a task management system. JIRA provides developers with the ability to open tickets for work tasks, schedule their completion, track progress, or announce its result at the end.

There are a few requirements for setting up JIRA. The first is to create an account on a separate JIRA account. You are creating a demo application so there is no requirement for you to connect to your regular task management software.

After creating an account you need to create a project from a Kanban board with the ability to create issues. With those two parts of JIRA set up there is nothing further for you to do with JIRA.

Paragon

With the JIRA account set up, the next step is for you to configure Paragon to handle Symbl.ai’s payload of action-items for configuring the creation of issues on the JIRA board automatically with Paragon’s POST request method.

Navigate from the Paragon portal select Paragon Connect. Inside of Paragon Connect select + New Integration. Within the + New Integrations select JIRA. After selecting the Connect button, the Connect button greys.

After selecting JIRA from the +New Integrations section, JIRA requires you to authenticate. After authenticating return to the +New Integrations. There you select + Create Workflow where you have the option to create a Trigger.

Under the menu that says “Choose a step to add,” select JIRA. Configure your step to Create Issue.

With JIRA integration set up the next step is to configure your JSON payload’s structure. The JSON payload’s structure is the JSON payload that is sent to the integration’s API through a POST request.

Configure a JSON payload structure in the following way:

{

"name": "action_item_name",

"userId": "action_item_userId",

"content": "action_item_content"

}

The result is that the JSON structure becomes the object that a method in the following format paragon.event("",{}); handles for the POST request.

paragon.event("upload_action_item",{

"name": "action_item_name",

"userId": "action_item_userId",

"content": "action_item_content"

});

With these two parts of Paragon configured, your next step is to configure an application to start handling events.

Streaming API

Symbl.ai’s API for its WebSockets is fully developed. Although a detailed explanation of best practices is found here, Symbl.ai’s Real-Time WebSocket API recommends as the following best practices for its adoption:

• Separate Channels per Person: audio for each person on a separate channel or over separate WebSocket connections for optimal results
• Lossless Codecs: FLAC or LINEAR16 codecs, if bandwidth is not an issue, or, if it is, Opus, AMR_WB, or Speex codecs.

Another great part of the Streaming API is that the Streaming API enables you to run its code in a Chrome browser without much trouble at all. If you want to read more about JavaScript WebSockets or Symbl.ai’s Streaming API, here are a few blogs:

• https://medium.com/codex/basic-javascript-code-for-live-conversational-ai-with-its-websockets-api-449d787ddd7a
• https://javascript.plainenglish.io/real-time-sentiment-analysis-api-in-javascript-with-symbl-ai-9e39d50f0654

Set up

Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret. With both of those you should authenticate either with a cURL command or with Postman so that you receive your x-api-key. Here is an example with cURL:

curl -k -X POST "https://api.symbl.ai/oauth2/token:generate" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d "{ \"type\": \"application\", \"appId\": \"<appId>\", \"appSecret\": \"<appSecret>\"}"

Ideally a token server would handle authentication (with code that makes RESTful API call for generating token) so that neither the appSecret nor the appId were ever exposed. However, cURL sets you up immediately anyway. With thex-api-key handy you are now ready to establish a WebSocket endpoint for performing live transcription.Run the WebSocket

• Deploy to a browser
• Open Console
• Witness live captioning of transcripts

Wrap the following lines of code to a file called index.js in method called openSocket():

var openSocket = async function() {
  const uniqueMeetingId = btoa('email@address.com');
  const accessToken = '';
  const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;const ws = new WebSocket(symblEndpoint);// Fired when a message is received from the WebSocket server
  ws.onmessage = (event) => {
    console.log(event);
  };// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
  ws.onerror  = (err) => {
    console.error(err);
  };// Fired when the WebSocket connection has been closed
  ws.onclose = (event) => {
    console.info('Connection to websocket closed');
  };// Fired when the connection succeeds.
  ws.onopen = (event) => {
    ws.send(JSON.stringify({
      type: 'start_request',
      meetingTitle: 'Websockets How-to', // Conversation name
      insightTypes: ['question', 'action_item'], // Will enable insight generation
      config: {
        confidenceThreshold: 0.5,
        languageCode: 'en-US',
        speechRecognition: {
          encoding: 'LINEAR16',
          sampleRateHertz: 44100,
        }
      },
      speaker: {
        userId: 'example@symbl.ai',
        name: 'Example Sample',
      }
    }));
  };const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });const handleSuccess = (stream) => {
    const AudioContext = window.AudioContext;
    const context = new AudioContext();
    const source = context.createMediaStreamSource(stream);
    const processor = context.createScriptProcessor(1024, 1, 1);
    const gainNode = context.createGain();
    source.connect(gainNode);
    gainNode.connect(processor);
    processor.connect(context.destination);
    processor.onaudioprocess = (e) => {
      // convert to 16-bit payload
      const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
      const targetBuffer = new Int16Array(inputData.length);
      for (let index = inputData.length; index > 0; index--) {
          targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
      }
      // Send to websocket
      if (ws.readyState === WebSocket.OPEN) {
        ws.send(targetBuffer.buffer);
      }
    };
  };handleSuccess(stream);
}

After creating your openSocket() method, create a page for rendering a button:

<!DOCTYPE html>
<html>

<head>
 <title>Parcel Sandbox</title>
 <meta charset="UTF-8" />

<script src="src/index.js">
 </script>
</head>

<body>

<div id="app"></div>

<button type="button" onclick="openSocket()">Click Me!</button>
 <div id="table-parent">
 </div>

</body>

</html>

Inside of the project’s root directory run python3 -m http.server 8000 so that you can run your application on local host: http://localhost:8000/. Run in Chrome. Press ⌥⌘J to open the console. You will run your Paragon Connect API code through the console there.

Link the Paragon Connect API

The final step after the Streaming API is up and running is to link the Streaming API to the Paragon Connect API.

Add the following snippet to your index.html:

<script

type="text/javascript"

src="https://cdn.useparagon.com/latest/sdk/index.js"

></script>

Run the following commands:

paragon.authenticate("<session-token>");

The method connects your application to Paragon Connect API. After authenticating your next step is to set up the connection to your integrated Third Party SaaS:

paragon.connect("jira", {
   type: "modal", 
   onSuccess: (event) => {}, 
   onFailure: (event) => {}
});

After both authenticating as well as connecting, the next step is to schedule the paragon.event method to sync with Symbl.ai’s conversation AI platform for detecting real-time action-items with JIRA.

paragon.event("upload_action_item",{

"name": "action_item_name",

"userId": "action_item_userId",

"content": "action_item_content"

});

Results

As you can see below, the Paragon Connect API sent the payload from Symbl.ai through a POST request to JIRA automatically right after the insight_response if-else statement captured an action-item.

As you can see below, the Paragon Connect API takes the JSON’s payload from Symbl.ai’s insight_response for insights detected as action-items logging those action-items into the JIRA board in real-time as the conversation over the WebSocket occurs.

What’s Next?

You completed your integration over a WebSocket protocol. However, there are several other protocols over which to voice automate the creation of real-time action-items from a Symbl.ai conversation. You can voice-automate over the Sessions Initiation Protocol, the Publicly Switched Telephone Network, the Real Time Messaging Protocol or over different technologies such as Node.js or different languages such as Python. Symbl.ai provides sample applications for all of these different protocols, technologies or languages!

After connecting with Paragon’s Connect API, the world is your oyster. You have the ability to create omni-channel, contextual, multi-channel conversations that extend beyond a voice, video, message conversation to such things as the Simple Mail Transfer protocol with an email provider or to the chat servers like Discord or Slack, both of which are integrated already in Paragon’s suite of API automations! You can create or update queries for real-time databases or servers like MongoDB or Firebase. The world is your oyster:

Community

Stuck? Free feel to ask us any questions on our Slack Channel or send us an email at developer@symbl.ai.

While ancient Greek pre-Socratic philosopher Heraclitus says, “τὰ πάντα ῥεῖ καὶ οὐδὲν μένει,” with many translating the phrases differently, at times emphasizing how the universe fluxes, at others how the flux is constant, the phrase’s meaning amounts more powerfully to delineating the inherent property of matter for movement against its complete absence in nothingness: Anything stationary is nothing. In terms crypto currency, nothing could be farther from the truth of its conversions from crypto to fiat. Cryptocurrencies as movement thus live accordingly.

Coindesk APIs

The Coindesk APIs asynchronously update the price by which a crypto currency converts into USD in realtime.

If, for instance, you make a call to the following API endpoint for current prices (i.e., https://api.coindesk.com/v1/bpi/currentprice.json) from Coindesk, the following .json data returns:

{
  "time": {
    "updated": "Mar 21, 2022 21:59:00 UTC",
    "updatedISO": "2022-03-21T21:59:00+00:00",
    "updateduk": "Mar 21, 2022 at 21:59 GMT"
  },
  "disclaimer": "This data was produced from the CoinDesk Bitcoin Price Index (USD). Non-USD currency data converted using hourly conversion rate from openexchangerates.org",
  "chartName": "Bitcoin",
  "bpi": {
    "USD": {
      "code": "USD",
      "symbol": "$",
      "rate": "41,307.2125",
      "description": "United States Dollar",
      "rate_float": 41307.2125
    },
    "GBP": {
      "code": "GBP",
      "symbol": "£",
      "rate": "31,370.2668",
      "description": "British Pound Sterling",
      "rate_float": 31370.2668
    },
    "EUR": {
      "code": "EUR",
      "symbol": "€",
      "rate": "37,497.4483",
      "description": "Euro",
      "rate_float": 37497.4483
    }
  }
}

Python Requests

Python does not support HTTP/S requests natively. The Requests library does. It facilitates the process of calling API endpoints with the lanugage. The first step to installing the Python Requests library is to ensure pip is installed in your Python environment.

pip3 install requests

With macOS you need to install pip with Homebrew. With a Linux kernel like Ubuntu, you install in the following way:

Step 1 — Updating the System

sudo apt-get update

Step 2 — Installing pip3

sudo apt-get -y install python3-pip

Step 3 — Verifying

pip3 --version

After pip is installed, install Python’s requests library.

pip3 install requests

Script

The next step is to create a Python script in your environment for accessing the Coindesk API.

#!/usr/bin/env python
#Bitcoin script
import requests
response = requests.get()
data = response.json()
print(data)

Within the requests.get() section of the function call for the variable for response, add the url : https://api.coindesk.com/v1/bpi/currentprice.json.

#!/usr/bin/env python
import requests
response = requests.get('https://api.coindesk.com/v1/bpi/currentprice.json')
data = response.json()
print(data)

The script’s first line declares the script to be so. The second is the keyword for importing the requests library. The third line response is a variable set to receive the results of a function call for the HTTP/S verb .get.

Parsing the data

The return data should have the following structure:

{'time': {'updated': 'Aug 31, 2024 22:23:01 UTC', 'updatedISO': '2024-08-31T22:23:01+00:00', 'updateduk': 'Aug 31, 2024 at 23:23 BST'}, 'disclaimer': 'This data was produced from the CoinDesk Bitcoin Price Index (USD). Non-USD currency data converted using hourly conversion rate from openexchangerates.org', 'chartName': 'Bitcoin', 'bpi': {'USD': {'code': 'USD', 'symbol': '$', 'rate': '59,107.08', 'description': 'United States Dollar', 'rate_float': 59107.0797}, 'GBP': {'code': 'GBP', 'symbol': '£', 'rate': '45,016.839', 'description': 'British Pound Sterling', 'rate_float': 45016.8385}, 'EUR': {'code': 'EUR', 'symbol': '€', 'rate': '53,436.819', 'description': 'Euro', 'rate_float': 53436.8193}}}

Since there is no documentation on Coinbase’s APIs, one way to understand the data is to loop through the URI’s return data for key value pairs. These key value pairs tell us which aspect of the URI’s return data return which pieces of information.

print("Print each key-value pair from JSON response")
for key, value in data.items():
    print(key, ":", value)
print(data)
print("Access directly using a JSON key name")

If we loop through the key value pairs, the nested keys are much easier to askance than if not.

time : {'updated': 'Aug 31, 2024 22:30:05 UTC', 'updatedISO': '2024-08-31T22:30:05+00:00', 'updateduk': 'Aug 31, 2024 at 23:30 BST'}
disclaimer : This data was produced from the CoinDesk Bitcoin Price Index (USD). Non-USD currency data converted using hourly conversion rate from openexchangerates.org
chartName : Bitcoin
bpi : {'USD': {'code': 'USD', 'symbol': '$', 'rate': '59,092.758', 'description': 'United States Dollar', 'rate_float': 59092.7582}, 'GBP': {'code': 'GBP', 'symbol': '£', 'rate': '45,004.217', 'description': 'British Pound Sterling', 'rate_float': 45004.2173}, 'EUR': {'code': 'EUR', 'symbol': '€', 'rate': '53,423.872', 'description': 'Euro', 'rate_float': 53423.8717}}

There are really only two key value pairs. These are time and bpi. Data from these two key value pairs provides both the time as well as the amount of Bitcoin in a specific currency. While the bpi provides both the pound sterling as well as the Euro, you’ll print the dollar amount. To print both the time as well as the dollar amount, let’s access the following nested keys.

print(data["time"]['updated'])
print(data['bpi']['USD']['rate'])

Depending upon how fancy you would like these pieces of data to be displayed, there are a number of other ways to configure how the data is read into the terminal. Without anything fancy, these should display as follows:

Aug 31, 2024 22:35:54 UTC
59,076.459

Conclusion

If you would like to modify the script to run a specific intervals, you can do so with a simple command such as schedule. An example of that modification command could be schedule.every().day.at("01:00").do(job,'It is 01:00')where the every method runs every at 1:00PM everyday (i.e., the close of market) where job is defined as the method call to the Coindesk API.

New to trading? Try crypto trading bots or copy trading

How to Program a Python Script to Track Bitcoin Automatically with Coindesk APIs & Coinbase APIs was originally published in Coinmonks on Medium, where people are continuing the conversation by highlighting and responding to this story.

Symbl.ai is a Conversation Intelligence API platform for developers to augment the speaking realities of conversations in real-time. Many of the realities that speakers face in conversations are limited by the current state of the art for voice, video, message, or broadcast platforms whose native functionality does not extend beyond enabling calls, streams, chats, or live events. Symbl.ai enables developers developing on any of the many streaming platforms to extend the functionalities of those platforms vastly through the introduction of no more than a few API calls.

Among the API calls with which you as a developer are empowered to extend the functionality of your calls is Symbl.ai’s Async API. With Symbl.ai’s Async API you have the power to augment the speaking realities of your speakers by separating their voices. With separated speakers, you are able to connect, transform or visualize the aspects of a conversation that are unique to the separated speaker.

Symbl.ai’s Python SDK. Symbl.ai’s Python SDK provides convenient methods for making calls to API endpoints like the one uploading a recorded call for processing with the Async API. In the following guide, you will upload a recorded call with Symbl.ai’s Async API with the API call configured to enable speaker separation but with Symbl.ai’s Python SDK.

How to implement speaker separation with Async Audio or Video Files | Symbl Docs

Sign up

Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret. With both of those you should authenticate either with a cURL command or with Postman so that you receive your x-api-key. Here is an example with cURL:

curl -k -X POST "https://api.symbl.ai/oauth2/token:generate" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d "{ \"type\": \"application\", \"appId\": \"<appId>\", \"appSecret\": \"<appSecret>\"}"

Ideally, a token server would handle authentication (with code that makes RESTful API call for generating token) so that neither the appSecret nor the appId were ever exposed. However, cURL sets you up immediately anyway. With thex-api-key handy you are now ready to establish a WebSocket endpoint for performing live transcription.

Setting Up Symbl.ai’s Python SDK 🐍

Symbl.ai’s Python SDK is a specially designed SDK to enable Pythonistas to program brand new experiences around conversations. In that respect Symbl.ai’s Python SDK empowers developers to access programmable intelligence directly in your mobile or web applications, client, or server.

Installation

Before you install the Symbl.ai Python SDK, please ensure that you have installed Python 2.7+ or Python 3.4+ (PyPy supported). Symbl.ai recommends you execute Python’s installation through Homebrew, the open-source package manager for macOS.

Homebrew

To install Homebrew, open Terminal or your favorite OS X terminal emulator and run:

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

After installing Homebrew, run the following command:

brew install python

After the installation, check to see whether your version of Python is compatible with Symbl.ai’s installation requirements:

python --version

If the version is higher than Python 2.7+ or Python 3.4+, then you’re all set to download Symbl.ai’s Python SDK.

Installing Symbl.ai’s Python SDK

Now that Python is installed, the next step is to install Symbl.ai’s Python SDK. To install Symbl.ai’s Python SDK, you install pip in the following way:

pip install --upgrade symbl

In a similar way, you install the same:

python -m pip install --upgrade symbl

Configuration

In a testing environment like the one you create to transform your conversation into a time-series graph, you run credentials in-line. Since you quickly test the current code, you can utilize the credentials variable to pass your appId and appSecret as a part of the code itself. However, you must add the credentials variable every time make a new API call.

import symbllocal_path = r'c:/Users/john/Downloads/business_meeting.mp3'# Process audio fileconversation_object = symbl.Audio.process_file(
file_path=local_path
credentials={app_id: <app_id>, app_secret: <app_secret>},  #This is optional if you didn’t setup the symbl.conf file in your home directory. 
)

Making a Call to the Async API with the Python SDK

To start the process of hacking your conversations with Symbl.ai’s Python SDK you import the Symbl SDK.

import symbl

The next step is to configure a payload in much the same way that you would configure a payload for an ordinary HTTP request in any one of your favorite programming languages. Here is a payload.

payload = {'url':'https://symbltestdata.s3.us-east-2.amazonaws.com/sample_audio_file.wav',}

After configuring the payload, create a dictionary for storing credentials in the following way:

credentials_dict = {'app_id': '<app_id>', 'app_secret': '<app_secret>'}

Last but not least create a conversation object. After the conversation object is created, the next step is to pipe the response selectively into the data you want to visualize in particular. The conversation object is a call to Symbl.ai’s Async API for processing audio through a URL.

conversation = symbl.Audio.process_url(https://medium.com/payload=payload,%20credentials=credentials_dict)

Parameters for Speaker Separation

To configure the conversation payload to deliver parameters directing the Async API to enable both a count as well as a separation of speakers, you configure a parameters dictionary with key-value pairs like so:

Note: If you made the call without specifying the number of speakers, you might have received the following message:

{"message":"diarizationSpeakerCount must be greater than zero and needs to be passed to accurately determine unique speakers in the audio"}

To properly configure the conversation payload, you have to enable both a count as well as a separation of speakers in the following way:

parameters={'enableSpeakerDiarization': True, 'diarizationSpeakerCount': 2}

Separate Speakers

By adding the following to your conversation payload, your proper configuration directs the Async API to both counts as well as separates the speakers in the recorded call.

With the proper configuration set up in your conversation payload, make your API call. After the API call is made, you receive a conversationId with which to check whether or not or how the numbered speakers were separated.

Conversation API’s Message API

After separating the speakers, number speaker labels appear within the messages of the Conversation API’s Message API’s return data. The data looks like the following:

{

"messages": [

{

"id": "5543101041999872",

"text": "But what?",

"from": {

"id": "f0fe6722-49c7-47c4-9708-03e4e5182508",

"name": "Speaker 2"

},

"startTime": "2021-08-16T19:38:01.747Z",

"endTime": "2021-08-16T19:38:02.147Z",

"conversationId": "0000-0000-0000-0000",

"phrases": []

}]

Numpy, Matplotlib, Pandas

Built on top of Numpy (i.e., multi-dimensional array objects) together with Matplotlib (i.e., capabilities for data visualizations), Pandas is a Python library for DataFrames, a programmatic object for two-dimensional, size-mutable, potentially heterogeneous tabular data (i.e., rectangular or tabular data of rows & columns or DataFrames). Pandas handle CRUD operations within those objects at the same time that they provide access to statistical analysis. With more than 14 million downloads a month, Pandas is the key that releases the genies in the bottles of Data Science throughout the world.

Before passing back to the analysis, there are two sets of important keywords to mention about Pandas. 1) The three components are accessible through attributes: .values — data values in a two-dimension NumPy array, .columns, and .index — data values in row numbers or row labels. In addition to these attributes, there are display methods. These are .head() — displays the first row, .info() — display the names of columns, and .shape() — number of rows & columns (attribute, not method, no ()).

One other convenience method called .describe() is used for calculating some statistical data like percentile, mean, and std of the numerical values of a Series or a DataFrame. It analyzes both numeric and object series and also the DataFrame column sets of mixed data types.

Creating Separate Numpy Arrays for Analyzing the Speakers Separately

The next step is to loop through the speaker-separated transcription with the conversationIdto create separate NumPy arrays for analyzing the speakers separately. Since Symbl.ai provides timestamped, speaker-separated data from the Message API within the Conversation API’s suite of API endpoints, your first step now is to perform an analysis of sentiments on your speaker separated timestamped data.

Message API & Sentiment Analysis

Symbl.ai’s Python SDK makes the process of performing an analysis of sentiments on your speaker separated timestamped data convenient. You make a call to more than a single API endpoint with a payload. A payload is a dictionary just like the one for speaker separation, except with a new key-value pair: parameters={‘sentiment’: True}.

response = conversation.get_messages(parameters={'sentiment': True})

After making the call to the conversation API through the .get_messages convenience method on the Conversation API’s Message API, you receive a JSON return data in the following format:

{

"messages": [

{

     "id": "5543101041999872",

     "text": "But what?",

    "from": {

         "id": "f0fe6722-49c7-47c4-9708-03e4e5182508",

         "name": "Speaker 2"

},

    "startTime": "2021-08-16T19:38:01.747Z",

    "endTime": "2021-08-16T19:38:02.147Z",

    "conversationId": "0000-0000-0000-0000",

    "phrases": [],

    "sentiment": {

        "polarity": {

        "score": -0.301

        },

        "suggested": "negative"

        }

}]

Loading the Numpy Arrays

The first step is to empty initialize a set of arrays.

timestamps1, timestamps2, polarities1, polarities2 = [], [], [], []

After having empty initialized these two empty arrays, your next step is to loop through the Message API’s return data to map both the timestamps as well as the sentiments correctly to the speaker-separated arrays.

As you may imagine, 1) your first step is to fetch the JSON data from the conversation payload. 2) Your second step is to create a Python if/else statement where the first condition is whether or not the speaker is speaker 1 while the second condition is whether or not the speaker is speaker 2. Based on which speaker is which, you configure the Python flow logic to map the speaker-separated timestamps to the NumPy array, the speaker-separated timestamps to the NumPy array.

timestamps1, timestamps2, polarities1, polarities2 = [], [], [], []
for message in response.messages:
    if message.from.name == "speaker 1":
        timestamps1.append(message.start_time)
        polarities1.append(message.sentiment.polarity.score)
    elif message.from.name == "speaker 2":
        timestamps2.append(message.start_time)
        polarities2.append(message.sentiment.polarity.score)

print(timestamps1, timestamps2)
print(polarities1, polarities2)

Creating Speaker Separated DataFrames with Pandas

The next step is to merely map the NumPy arrays directly to Pandas’ DataFrames in the following way:

import pandas as pd

df1 = pd.DataFrame({"timestamps": timestamps1,"polarities": polarities1})

df2 = pd.DataFrame({"timestamps": timestamps2,"polarities": polarities2})

Plotting Multiple DataFrames with Matplotlib to a Single Plot: pandas.DataFrame.plot()

To perform a simultaneous time series analysis of the timestamped, speaker separated data from the Message API, you load the DataFrames directly into a .plot call on Matplotlib, configuring the x-axis to display the timestamps, the y-axis to display the polarities.

You might think that the logical code implementation for that type of configuration is to run both DataFrames through the same method call.

df1.plot(x="timestamps", y="polarities")
df2.plot(x="timestamps", y="polarities")

You might think that the logical code implementation for that type of configuration is to run both DataFrames through the same method call.

The logical code implementation, however, is quite different in Pandas. You make the assignment of one DataFrame’s plot equivalent to the other in the following way:

ax = df1.plot()

df2.plot(ax=ax)

What’s Next

In addition to adding bells and whistles to your graphs like labels, you may consider how Symbl.ai’s Python SDK provides access to many of Symbl.ai’s APIs. In particular, you can enable Symbl.ai’s Python SDK on the real-time Telephony API (i.e., Telephony API (in Real-time) or any one of the Async APIs for voice, video, or messaging. A complete list is here:

• Async Text API
• Async Audio (File) API
• Async Audio (URL) API
• Async Video (File) API
• Async Video (URL) API

In addition, Symbl.ai’s Python SDK contains links to enable Symbl.ai on its Telephony APIs with Python special methods. The methods for dialing in to a call, work on the Session Initiation Protocol (SIP). With Symbl.ai’s Python SDK dialed into a call, you are in a position to expand the experiences around calls in real-time. After connecting to a SIP call in real-time, you subscribe to Symbl.ai’s events such as events for contextual insights like follow-ups, questions, or action-items, or you create a request to deliver a post-meeting summary directly to your inbox.

Community

Symbl.ai’s invites developers to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai's GitHub.

More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter and LinkedIn. Join our community Discord.

Perform a Timeseries Analysis of Separated Speakers from Recorded Calls with Symbl.ai’s Python SDK was originally published in Python in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

Pipedream

In pursuit of that goal is Pipedream. Pipedream is a serverless, real-time API platform for programmatically transforming data from one state to another without a human hand. It provides multiple points of entry for Third Party Software as a Service to enable developers to connect APIs remarkably fast with triggers or actions in Node.js. With managed authentication for 300+ OAuth or key-based APIs, Pipedream makes environment variables accessible for credentials or references in any workflow. It is serverless so the platform does not require developers to deploy their automation engines to the cloud.

Since Pipedream works with any or all APIs, Pipedream’s platform automates the process of connecting Telnyx PSTN calls seamlessly. Symbl.ai is already enabled Pipedream. Pipedream therefore empowers developers to combine Symbl.ai’s Conversation Intelligence API platform with Telnyx to create powerful automations that save time, money, or effort throughout the entire process of a call.

In your automation you program Pipedream to enable call recording through the interpretation of a WebHook’s response data. After automating the recording, you enable Symbl.ai to process the recorded call upon completion. You literally daisy chain the three together, Telnyx, Pipedream, Symbl.ai.

Prerequisites

• Pipedream Account
• Telnyx Account
• Symbl.ai Account
• cURL

Telnyx

Telnyx is a real-time communications platform with full, feature-rich voice functionalities, making it quick and easy to set up and port numbers around the world, configure messaging, control VoIP and IP network functions, and define how and where communications can be used in real time. The Telnyx API can power a host of specialty applications, from call tracking to cloud-based PBX, dynamic security, and authentication use cases.

Telnyx

Telnyx’s Voice APIs provide you as a developer with access to recorded calls through a single API call.

Steps

To analyze a recorded call with Telnyx, you set up your Voice API configuration with Telnyx’s Call Control within the Telnyx Portal. Just follow these simple steps:

1. Sign up for a Telnyx account.
2. Create an application to configure how you connect your calls.
3. Buy or port a phone number to receive inbound calls, and assign this number to your new application.
4. Create an outbound voice profile to make outbound calls, and assign your application to this new profile.
5. Record the call
6. Generate a Public URL for recorded calls

These steps are summarized on Telnyx’s quickstart guide.

Portal Signup

Let’s begin with the easiest step of all. Head to the signup page to sign up for your free Telnyx account. It’ll give you access to our Mission Control Portal, where you can buy numbers, set up and manage connections and a lot more. The first step is to search and purchase a Telnyx Voice API number.

Search and Purchase

After clinking on numbers, you are provided with the ability to search and buy a number:

After selecting a number, checkout.

Editing your Call Control App

After selecting a number, you navigate to the Call Control App.

After clicking “Create Your First Application”, you edit your Call Control App.

There are three steps to take in editing the Call Control App:

1. Create an App Name.
2. Copy of the Call Control Application number as later the number is used in a cURL command as a uuid.
3. Paste the link you called your “Your unique URL.”

With those three steps out of the way, your Call Control App is ready to enable outbound calls through POST requests with cURL commands.

The First cURL Request

To initiate a Telnyx Voice API call for recording, you make the following POST request:

curl -X POST \
  --header "Content-Type: application/json" \
  --header "Accept: application/json" \
  --header "Authorization: Bearer YOUR_API_KEY" \
  --data '{"connection_id": "UUID", "to": "YOUR TELEPHONE NUMBER", "from": "YOUR TELNYX VOICE API NUMBER"}' \
  https://api.telnyx.com/v2/calls

As you can see above, you will need to change the variables for the following:

1. YOUR_API_KEY
2. YOUR_TELEPHONE_NUMBER
3. YOUR_TELNYX_VOICE_API_NUMBER
4. UUID

The UUID is the Call Control Application number from number 2 in the section above.

After the request successfully POSTs, the webhook server returns a POST request with return data like so:

{
  "data": {
    "event_type": "call.initiated",
    "id": "3d4d3f3a-cfd6-437e-9592-7554a9996a45",
    "occurred_at": "2021-09-14T18:52:40.155477Z",
    "payload": {
      "call_control_id": "ef8ffc44-158c-11ec-8de2-02420a0d4168",
      "call_leg_id": "ef8ffc44-158c-11ec-8de2-02420a0d4168",
      "call_session_id": "ef8ab3d8-158c-11ec-b771-02420a0d4168",
      "client_state": null,
      "connection_id": "1725531088727573914",
      "direction": "outgoing",
      "from": "+16182216200",
      "state": "bridging",
      "to": "+19168739475"
    },
    "record_type": "event"
  },
  "meta": {
    "attempt": 1,
    "delivered_to": "https://YOURPIPEDREAMWEBHOOK.pipedream.net"
  }
}

The Second cURL Request

To initiate the recording for Telnyx’s Voice API call, you execute the following POST request withcURL:

curl -X POST \
  --header "Content-Type: application/json" \
  --header "Accept: application/json" \
  --header "Authorization: Bearer YOUR_API_KEY" \
  --data '{"format": "mp3", "channels": "single"}' \
  https://api.telnyx.com/v2/calls/{call_control_id}/actions/record_start

As you can see above, you will need to change the variables for the following:

1. YOUR_API_KEY
2. call_control_id

The call_control_id comes directly from the return data from the last POST request.

After the request successfully POSTs, the terminal returns the following data:

{
 "data": {
  "record_type": "recording",
  "recording_id": "c84048c4-e4cd-48f6-a94e-cf8be3fbd884",
  "result": "ok"
}

The next step is to hang up the call. With the call hung up, the webhook server serves the public URL for accessing the recorded call.

Public URLs for Recorded Calls

After recording a call with Telnyx’s Voice API, you make a call with one of the call commands to expose the webhook for the public recording URL (e.g., public_recording_urls) in the call.recording.saved object.

After successfully making the call, the return data POSTs to the webhook server. Within the return data for the JSON, the recording_urls looks like the following before it is posted to the web hook:

{
 "data": {
  "event_type": "call.recording.saved",
  "id": "0ccc7b54-4df3-4bca-a65a-3da1ecc777f0",
  "occurred_at": "2018-02-02T22:25:27.521992Z",
  "payload": {
   "call_leg_id": "428c31b6-7af4-4bcb-b7f5-5013ef9657c1",
   "call_session_id": "428c31b6-abf3-3bc1-b7f4-5013ef9657c1",
   "channels": "single",
   "client_state": "aGF2ZSBhIG5pY2UgZGF5ID1d",
   "connection_id": "7267xxxxxxxxxxxxxx",
   "public_recording_urls": {
    "mp3": "http://example.com/recording.mp3",
    "wav": "http://example.com/recording.wav"
   },
   "recording_ended_at": "2018-02-02T22:25:27.521992Z",
   "recording_started_at": "2018-02-02T22:20:27.521992Z",
   "recording_urls": {
    "mp3": "http://example.com/recording.mp3",
    "wav": "http://example.com/recording.wav"
   }
  },
  "record_type": "event"
 }
}

After posting to the webhook, the recording_urls looks like the following before it is posted to the web hook:

{
  "data": {
    "event_type": "call.recording.saved",
    "id": "47ee6238-6c0d-4f49-bd46-b2b00ea42fbd",
    "occurred_at": "2021-09-14T18:54:58.177200Z",
    "payload": {
      "call_control_id": "ef8ffc44-158c-11ec-8de2-02420a0d4168",
      "call_leg_id": "ef8ffc44-158c-11ec-8de2-02420a0d4168",
      "call_session_id": "ef8ab3d8-158c-11ec-b771-02420a0d4168",
      "channels": "single",
      "client_state": null,
      "connection_id": "1725531088727573914",
      "end_time": "2021-09-14T18:54:57.517467Z",
      "format": "mp3",
      "public_recording_urls": {},
      "recording_ended_at": "2021-09-14T18:54:57.517467Z",
      "recording_id": "c84048c4-e4cd-48f6-a94e-cf8be3fbd884",
      "recording_started_at": "2021-09-14T18:54:38.008036Z",
      "recording_urls": {
        "mp3": "https://s3.amazonaws.com/telephony-recorder-prod/b880fcdb-d93b-46e9-9c62-2aa7c5b2eb06/2021-09-14/ef8ffc44-158c-11ec-8de2-02420a0d4168-1631645677.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIASDC5E2MJDPKMH7UW%2F20210914%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210914T185458Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=b586b5ed26263edb7fd13c7b227c094e4ee9dd45403278d6f9335ee034c4d65d"
      },
      "start_time": "2021-09-14T18:54:38.008036Z"
    },
    "record_type": "event"
  },
  "meta": {
    "attempt": 1,
    "delivered_to": "https://YOURPIPEDREAMWEBHOOK.pipedream.net"
  }
}

Symbl.ai requires nothing more than that URL to handle analyzing the Telnyx recorded call. The great thing about Telnyx is that Telnyx empowers developers to provide options with which to determine the file type of the recorded call hosted on the S3 bucket. In recording_urls above the file type is .mp3. Symbl.ai’s Conversation Intelligence API platform accepts .mp3 files right out of the box.

Pipedream

Pipedream is one of the most developer friendly platforms for automations. It greets developers with a special onboarding immediately upon login. In Pipedream’s special onboarding, Pipedream teaches you how to create an HTTP triggered workflow where you 1) trigger a workflow on an HTTP request, 2) retrieve data from an API using Node.js with npm, 3) deploy your changes for testing.

Automation Process for Pipedream with Telnyx

Pipedream initiates its automation after the enabled WebHook server returns the response data from the final POST request in the chain of POST requests made to create a recorded call. The chain is 1) v2/calls, 2) v2/calls/{call_control_id}/actions/record_start where the response data to the latter call is an .mp3 file hosted by AWS from the section on recording_urls. You access that data from the data pipeline in the following way: event.body.data.recording_urls. Accordingly, the process by which you automate the analysis of recorded call is a multi-step process so after you have automated the call recording, the next step is to create another automation for transferring the recorded call to Symbl.ai for processing.

Automating Telnyx Call Recording with Symbl.ai within Pipedream

After familiarizing yourself with Pipedream, the next step is to create a workflow. A workflow is a set of steps designed to respond to events over a webhook. In your workflow for automating call recording, you configure two steps, one for a trigger, another for processing.

Environment Variables

However, before creating a workflow, you need to configure your environment variables appropriately. Navigate to Settings > Environment Variables > New Environment Variable. For YOUR_ENV_VAR_NAME, set the name as TELNYX_API_KEY, add the value. After adding the value, the next step is to add your Symbl.ai appId and appSecret with their respective values for each.

The Event Flow

Pipedream is a reactive platform that reacts to events in real-time. Telnyx, in the same regard, is reactive, an event driven architecture where every action creates an opposite and equal reaction. In that reactive system, Telnyx creates events when a call is initiated (i.e., call.initiated), hung up (i.e., call.hangup), answered (i.e., call.answered), or recorded (i.e., call.recording.saved) for each of which Telnyx issues a POST request on its webhook. The request / response model, common among all APIs of the world, is a basic HTTP framework in motion.

Pipedream WebHook

Here is where both Pipedream as well as Telnyx are exceptionally developer friendly. Pipedream creates a link for a webhook whereas Telnyx receives the created links for webhooks. One fits into the other like as in the two go together like hand and glove. The next step is to take your Pipedream webhook so that you can add it to Telnyx.

1. Login into the Telnyx platform
2. Navigate from the Dashboard to Call Control / Applications
3. Select your number.
4. Navigate from Edit your Call Control App > Send a webhook to the URL.
5. Paste your Pipedream webhook url: “delivered_to”: “https://YOURPIPEDREAMWEBHOOK.pipedream.net"

After you have added your Pipedream WebHook url, close the Telnyx account. With the WebHook set up in the Telnyx platform, the next step is to create the workflow for responding to the events the Telnyx APIs POST to the Pipedream WebHook.

Steps for Trigger

The first step is to set up a trigger. A trigger in the Pipedream platform is a step in the portal where the events posted to the webhook are handled. The workflow triggers the workflow on each request. The advantage that Pipedream provides is that the platform’s triggers not only handle the event, they handle the event’s JSON return data, parsing it entirely upon receipt. In terms of Telnyx, the webhook therefore parses the return data for the Telnyx events mentioned earlier: (i.e., call.initiated), hung up (i.e., call.hangup), answered (i.e., call.answered), or recorded (i.e., call.recording.saved).

Triggering

Pipedream’s own developer onborading teaches you how to properly configure an HTTP API as a trigger. Pipedream generates a unique URL to trigger :(i.e., your https://YOURPIPEDREAMWEBHOOK.pipedream.net). Since Pipedream automatically detects the JSON schema of the HTTP API’s return data, Pipedream provides developers with a readily made solution for enabling developers to gain access to the data contained in the step: steps.trigger.event. Consequently, you access the return data from that single instance (i.e., .event).

Processing with Node.js

In the next step, which is a sandbox for Node.js, Pipedream feeds the events or steps into a large async call (i.e., an invention of genius for its simplicity). In the async step, the Node.js app completes the calls to functions for handling the different types of functions with which to program automatic reactions. The async function call looks like the following:

async (event, steps) => {}

In the async call declare the following:

const axios = require('axios')

const name = "Pipedream's Automation for Telnyx with Symbl.ai"

const type = event.body.data.event_type

In the next step you need to program the Telnyx events mentioned earlier: (i.e., call.initiated), hung up (i.e., call.hangup), answered (i.e., call.answered), or recorded (i.e., call.recording.saved) in the following way:

After you have identified the async function call, the next step is to program a type to toggle, as you toggle from one type of event to another. In the following, you toggle from one type of Telnyx event to another:

const type = event.body.data.event_type

if (type === "call.recording.saved") {

console.log("The Call is Saved")

} else if (type === "call.answered") {

console.log("The Call is Answered")

const callControlId = event.body.data.payload.call_control_id

const recordingInitiated = await startRecording(callControlId)

console.log("The recording is initiated:", recordingInitiated)

} else if (type === "call.initiated" || type === "call.hangup") {

$end()

} else  {

$end()

}

The code there is straight forward an if/else statement for toggling on different types of events. The next step is to program the reactions to be triggered in regards to different types of events. The first reaction is to trigger recording when the call is answered.

const startRecording = async (callControlId) => {

const data = {

format: 'mp3',

channels: 'single'

}

try {

const response = await axios({

method: 'POST',

url: `https://api.telnyx.com/v2/calls/${callControlId}/actions/record_start`,

headers: {Authorization: `Bearer ${process.env.TELNYX_API_KEY}`},

data: data

})

return response.data

} catch (error) {

console.log(error)

}

}

After a call is answered, the callControlId is returned. With the callControlId returned, the next step is to make a POST request to Telnyx’s API endpoint for recording calls: /actions/record_start. After the function for enabling call recording is configured, the next step is designed to save the call for processing.

if (type === "call.recording.saved") {

// Save the url from the event for recording

const url = event.body.data.payload.recording_urls.mp3

//Submit Job

const token = await symblToken()

const job = await pushAudioUrl(url, token)

console.log(job)

}

The next step is to program the functions called to trigger both Symbl.ai’s authentication as well as its audio processing.

const symblToken = async () => {

const data = {

type: "application",

appId: process.env.SYMBL_APPID,

appSecret: process.env.SYMBL_SECRET

}

try {

const response = await axios({

method: 'POST' ,

url: 'https://api.symbl.ai/oauth2/token:generate',

headers: {'Content-Type': 'application/json'},

data: data

})

return response.data.accessToken

} catch (error) {

console.log(error)

}

}

The function symblToken() is an HTTP POST request that passes the appId and appSecret over to Symbl.a’s authentication API, returning a token.

The next function is an HTTP POST request to Symbl.ai’s audio processing API. API sends the url from event.body.data.payload.recording_urls.mp3 over to Symbl.ai for processing.

const pushAudioUrl = async (url, token) => {

const data = {

name: name,

url: url,

webhookUrl: 'https://YOURPIPEDREAMWEBHOOK.pipedream.net'

}

try {

const response = await axios({

method: 'POST',

url: 'https://api.symbl.ai/v1/process/audio/url',

headers: {

'Content-Type': 'application/json',

Authorization: `Bearer ${token}`

},

data: data

})

return response.data

} catch (error) {

console.log(error)

}

}

Upon Completion

After the automation is complete, you receive both a conversationId as well as a jobId with which to check the results you have achieved through the automation.

Community

Symbl.ai’s invites developers to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai's GitHub.

Symbl.ai’s developer platform for programmable intelligence derived from voice, video, message, or broadcast streams, empowers developers to transform experiences around conversations digitally. In the following guide, you use Symbl.ai’s Python SDK, Message API to take sentiment analysis to the next level of digital transformation with time-series analyses.

What is Symbl.ai?

Symbl.ai enables developers take their user enabled, remote, virtual voice, video, chat, or broadcast streaming ecosystems to the next level of digital transformation. Digitally transforming Real-Time Communications (RTC) enables developers to transform conversations.

Symbl.ai’s proprietary algorithms simplify conversations so that conversation data such as topics, insight types like questions, follow-ups, or action-items, or sentiments are available for big data manipulation with digital transformation.

With the ability to program the intelligence you receive from your conversations, your ability to create schemes for enhancing real-time engagements widens to beyond mere transcription to greater connectivity, digital transformations or human readable visualizations.

How to Get Up and Running with Symbl.ai

Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret. With both of those you should authenticate either with a cURL command or with Postman so that you receive your x-api-key. Here is an example with cURL:

curl -k -X POST "https://api.symbl.ai/oauth2/token:generate" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d "{ \"type\": \"application\", \"appId\": \"<appId>\", \"appSecret\": \"<appSecret>\"}"

Ideally a token server would handle authentication (with code that makes RESTful API call for generating token) so that neither the appSecret nor the appId were ever exposed. However, cURL sets you up immediately anyway. With thex-api-key handy you are now ready to establish a WebSocket endpoint for performing live transcription.

Python and Symbl.ai’s Python SDK 🐍

Symbl.ai’s Python SDK is a specially designed SDK for enable Pythonistas to program brand new experiences around conversations. In that respect Symbl.ai’s Python SDK empowers developers to access programmable intelligence directly in your mobile or web applications, client or server.

Installation

Before you install the Symbl.ai Python SDK, please ensure that you have installed Python 2.7+ or Python 3.4+ (PyPy supported). Symbl.ai recommends you execute Python’s installation through Homebrew, the open source package manager for MacOS.

Homebrew

To install Homebrew, open Terminal or your favorite OS X terminal emulator and run:

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

After installing Homebrew, run the following command:

brew install python

After the installation, check to see whether your version of Python is compatible with Symbl.ai’s installation requirements:

python --version

If the version is higher than Python 2.7+ or Python 3.4+, then you’re all set to download Symbl.ai’s Python SDK.

Installing Symbl.ai’s Python SDK

Now that Python is installed, the next step is to install Symbl.ai’s Python SDK. To install Symbl.ai’s Python SDK, you install pip in the following way:

pip install --upgrade symbl

In a similar way, you install the same:

python -m pip install --upgrade symbl

Configuration

In a testing environment like the one you create to transform your conversation into a time series graph, you run credentials in-line. Since you quickly test the current code, you can utilize the credentials variable to pass your appId and appSecret as a part of the code itself. However, you must add the credentials variable every time make a new API call.

import symbl

local_path = r'c:/Users/john/Downloads/business_meeting.mp3'

# Process audio file

conversation_object = symbl.Audio.process_file(
file_path=local_path
credentials={app_id: <app_id>, app_secret: <app_secret>},  

#This is optional if you didn’t setup the symbl.conf file in your home directory. 
)

Using the Python SDK

To start the process of hacking your conversations with Symbl.ai’s Python SDK you import the Symbl SDK.

import symbl

The next step is to configure a payload in much the same way that you would configure a payload for an ordinary HTTP request in any one of your favorite programming languages. Here is a payload.

payload = {

'url':'https://symbltestdata.s3.us-east-2.amazonaws.com/sample_audio_file.wav',

}

After configuring the payload, create a dictionary for storing credentials in the following way:

credentials_dict = {'app_id': '<app_id>', 'app_secret': '<app_secret>'}

Last but not least create a conversation object. After the conversation object is created, the next step is to pipe the response selectively into the data you want to visualize in particular. The conversation object is a call to Symbl.ai’s Async API for processing audio through a URL.

conversation = symbl.Audio.process_url(https://medium.com/payload=payload,%20credentials=credentials_dict)

Making a Call to the Conversation API’s Message API

To fetch the sentiments on the messages from the conversation, you make a call to the Conversation API’s Message API with a parameter. Below is the code.

response = conversation.get_messages(parameters={'sentiment': True})

After fetching the sentiments on the messages from the conversation, you create two empty arrays. The first one is timestamps, where Symbl.ai’s data science algorithms prepares the data for a clocked index; the second one is polarities where Symbl.ai’s data science algorithms evaluates the sentiments on the words in the message to return a score.

timestamps, polarities = [], []

Since your goal is to create a human readable interpretation of a timeseries analysis of sentiments on messages, you create two arrays, the first corresponding to the y-axis, the second to the x-axis. After creating these arrays, you save these arrays.

Pandas

The next step in the process is to create DataFrames out of your arrays of data (i.e., timestamps and polarities) with Python’s Pandas library.

The first step to creating DataFrames from Python’s Pandas library is import the library.

import pandas as pd

After importing the library, the next step is to set the DataFrame’s data with the values from the two arrays. To do so, declare the following:

df = pd.DataFrame({"timestamps": timestamps,"polarities": polarities})

Matplotlib

After pitching your arrays of data in Pandas’ DataFrames, your next step is to create graphs of the DataFrames with Python’s Matplotlib library.

The first step is to import the Matplotlib. To do so, declare the following:

import matplotlib as plt

The next step is to plot the DataFrame by setting up the x and y axes of the graph.

df.plot(x="timestamps", y="polarities")

Run the Code

If you successfully integrated your Python code snippets into your code editor, then your code editor displays a graph mapping the timestamps to the y-axis, the sentiments to the x-axis. Below is a screenshot of the finished product.

The graph reads much more easily than the sentiment’s polarity scores. Apart from attempting to infer whether a polarity score is up or down, your polarity scores are mapped to their respective timestamp. The respective timestamp is mapped to the polarity scores. In that way the rise and fall of polarities is much easier ‘to see’ than merely crunching numbers. It is a readily understand, human readable, colorful visualization.

What’s Next

In addition to adding bells and whistles to your graphs like labels, you may consider how Symbl.ai’s Python SDK provides access to many of Symbl.ai’s APIs. In particular, you can enable Symbl.ai’s Python SDK on the real-time Telephony API (i.e., Telephony API (in Real-time) or any one of the Async APIs for voice, video, or messaging. A complete list is here:

• Async Text API
• Async Audio (File) API
• Async Audio (URL) API
• Async Video (File) API
• Async Video (URL) API

In addition, Symbl.ai’s Python SDK contains links to enable Symbl.ai on its Telephony APIs with Python special methods. The methods for dialing into a call, work on the Session Initiation Protocol (SIP). With Symbl.ai’s Python SDK dialed into a call, your in a position to expand the experiences around calls in real-time. After connecting to a SIP call in real-time, you subscribe to Symbl.ai’s events such as events for contextual insights like follow-ups, questions, or action-items, or you create a request to deliver a post-meeting summary directly to your inbox.

Community

Symbl.ai’s invites developers to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai's GitHub.

More content at plainenglish.io

Timeseries Analysis of Calls with Symbl.ai’s Python SDK, Pandas, & Matplotlib was originally published in Python in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

Symbl.ai is non-assumptive, non-invasive, fully extensible, and has a user-friendly API for sentiment analysis that provides insights into the polarity of messages in real-time. Let’s look at what this all means.

1. Non-assumptive

The API is non-assumptive in that results for polarity are not relayed as a particular emotion but as a value w.r.t. negativity or positivity. It doesn’t assume an emotion.

2. Non-invasive

The API is non-invasive, since the result of the analysis is from the completed message, of which a conversation is composed, rather than a voice, a highly personal, non-revocable biometric aspect of human identity.

3. Fully extensible

The API is extensible in that the analysis, which may be as to the conversation itself, its speakers, or their own individual or aggregate messages, is fully programmable. You as a developer decide how create statistics or data around the values derived from sentiments anytime anywhere.

4. User-friendly API

Users, whose aver quantification do not surrender of their unique, individual ‘credentials’, may have no fear any aspect of Symbl.ai’s sentiment analysis, making the API extremely friendly on the user end. It is not a tone analyzer, for instance.

WebSockets for Symbl.ai

WebSockets are a computer protocol for communications as a vast improvement over the request/response model of HTTP. They provide full-duplex channels over a single Transmission Control Protocol (TCP) connection. WebSockets facilitate communications between clients or serves in real-time without the connection suffering from racy, high-latency, and bandwidth-intensive exchange under HTTP. Specified as ws or wss, unencrypted or encrypted, a WebSocket endpoint is a persistent bi-directional channel of communication without any overhead from HTTP residuals such as headers, cookies or artifacts. Here is Symbl.ai’s API for WebSockets:

wss://api.symbl.ai/v1/realtime/

Symbl.ai’s JavaScript WebSocket’s API

In the following blog you will enable Symbl.ai’s JavaScript WebSocket’s API to transcribe live in your browser. To enable live transcription with Symbl.ai’s JavaScript WebSocket’s API in the browser, you: 1) set up your account, 2) configure the WebSocket’s endpoint, 3) run the code in the browser, 4) transcribe live. After setting up the live transcription, you implement Symbl.ai’s non-assumptive, non-invasive, fully extensible, user-friendly API for sentiment analysis.

Set up

Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret. With both of those you should authenticate either with a cURL command or with Postman so that you receive your x-api-key. Here is an example with cURL:

curl -k -X POST "https://api.symbl.ai/oauth2/token:generate" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d "{ \"type\": \"application\", \"appId\": \"<appId>\", \"appSecret\": \"<appSecret>\"}"

Ideally a token server would handle authentication (with code that makes RESTful API call for generating token) so that neither the appSecret nor the appId were ever exposed. However, cURL sets you up immediately anyway. With thex-api-key handy you are now ready to establish a WebSocket endpoint for performing live transcription.

WebSocket’s Endpoint

To enable the WebSocket, you configure two values as query parameters that are fed directly into the WebSocket API’s endpoint. In turn you feed the WebSocket API’s endpoint directly into JavaScript’s own software for enabling real-time conversations.

Here are the first two values:

const uniqueMeetingId = btoa('EMAIL@ADDRESS.COM');
const accessToken = '';

With these two values for the WebSocket’s API endpoint set, you feed these directly into the WebSocket API’s endpoint:

const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}

If you want to test your WebSocket before any further integration, you load that endpoint with the data for uniqueMeetingId together with the accessToken into Hoppscotch.io, a free, fast, sleek API request builder.

Creating an Instance of JavaScript’s native WebSocket API

The next step is to create an instance of JavaScript’s native WebSocket API:

const ws = new WebSocket(symblEndpoint);

On that instance, you call methods specific to handling live transcription.

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  console.log(event);
};

// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
  console.error(err);
};

// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};

// Fired when the connection succeeds.
ws.onopen = (event) => {
  ws.send(JSON.stringify({
    type: 'start_request',
    meetingTitle: 'Websockets How-to', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
      }
    },
    speaker: {
      userId: 'example@symbl.ai',
      name: 'Example Sample',
    }
  }));
};

To set up the stream for accessing a user’s media devices such as their laptop’s microphone, program the brower’s navigator accordingly:

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });

The code above asks the user for permission to access his or her device through a pop-up. In mobile applications permission for access to devices is hardcoded into the controlling directory of an application binary such as in the manifest for Android of the property list for iOS. JavaScript doesn’t compile so there is no need to ask for permission so that you can ask for permission!

To set up the stream to handle events, program the following:

const handleSuccess = (stream) => {
  const AudioContext = window.AudioContext;
  const context = new AudioContext();
  const source = context.createMediaStreamSource(stream);
  const processor = context.createScriptProcessor(1024, 1, 1);
  const gainNode = context.createGain();
  source.connect(gainNode);
  gainNode.connect(processor);
  processor.connect(context.destination);
  processor.onaudioprocess = (e) => {
    // convert to 16-bit payload
    const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
    const targetBuffer = new Int16Array(inputData.length);
    for (let index = inputData.length; index > 0; index--) {
        targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
    }
    // Send to websocket
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(targetBuffer.buffer);
    }
  };
};

Trying Out the the Code

It might be a great idea to try out your WebSocket’s connection right now. After running your code directly in the browser, then you see the console logging events in real-time as you speak.

Stopping the Conversation

To stop your live transcription, you close your browsers or you send a stop_request directly to the server from within the browser’s console.

ws.send(JSON.stringify({
  "type": "stop_request"
}));

Real-Time Sentiment Analysis

Within the ws method for onmessage program the API endpoint to capture the polarity of Symbl.ai’s conversation’s messages, logging the value to through the console.

The first step is to capture the conversationId as a constant. You capture the conversationId through the parsed data that comes from the WebSocket’s API’s onmessage method.

const conversationId = data.message.data.conversationId;

The second step is to create the request. There are multiple libraries for handling HTTP requests so feel free to refactor theXMLHttpRequest here.

const request = new XMLHttpRequest();

The third step is to configure the request, setting the response’s type to text.

request.responseType = "text";

The fourth step is to configure the endpoint:

const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;

The fifth step is to set the request.

request.open("GET", sentimentEndpoint)

The sixth step is to set the request’s headers.

request.setRequestHeader('Authorization', `Bearer ${accessToken}`);
request.setRequestHeader('Content-Type', 'application/json');

The seventh step is to create a function to log the sentiment’s into the console.

request.onreadystatechange=(e)=> {
      console.log(request.responseText)
    }

The last step is send the request.

request.send()

Here is the fully programmable sentiment analysis API implemented in your ws method for onmessage in your WebSocket:

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  // You can find the conversationId in event.message.data.conversationId;
  const data = JSON.parse(event.data);
  if (data.type === 'message' && data.message.hasOwnProperty('data')) {
    console.log('conversationId', data.message.data.conversationId);
    const conversationId = data.message.data.conversationId;
    console.log('onmessage event', event);
    // You can log sentiments on messages from data.message.data.conversationId 
    const request = new XMLHttpRequest();
    request.responseType = "text";
    const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;
    request.open("GET", sentimentEndpoint)
    request.setRequestHeader('Authorization', `Bearer ${accessToken}`);
    request.setRequestHeader('Content-Type', 'application/json');
    request.onreadystatechange=(e)=> {
      console.log(request.responseText)
    }
    request.send()
  }
};

If your console logged sentiments upon a change in the state of events, then your code delivered real-time sentiment analysis in API!

What’s Next?

A Symbl.ai conversation configured to relay messages over a real-time websocket with its sentiment analysis API is neither the beginning nor the end of conversation data.

If you look closely at the data from your live WebSocket connection, the conversationId may be applied to new API calls to access to more AI insights for action items, topics, or analytics. Within a fully programmable Symbl.ai conversation you receive not just sentiment analysis but access to an analytics API that provides access to the following conversation data:

1. Speaker Ratio — Speaker’s talk and listen ratio and time.
2. Talk Time — Overall duration of the conversation.
3. Silence — Overall duration of silence.
4. Pace — Words per minute spoken in the conversation.
5. Overlap — When more than 1 speaker are speaking the same time, then conversation has overlap.

With Speaker Ratio, Talk Time, Silence, Pace, or Overlap, you have the ability to create application logic or notifications that augment conversational experiences to a step far beyond merely transcribing a speaker or speakers in a real-time speech-to-text application or automating speech recognition through no more than a single a API endpoint.

Since real-time sentiment analysis is based on messages rather than tonality, Symbl.ai’s API for topics, which aggregates the most salient subjects of a conversation in a set of word groups called ‘topics’, are fully programmable as data on messages not unlike the data on messages for speaker events.

Analytics into data for sentiment or speaker events in general is but a part of Symbl.ai’s extensible, fully programmable, developer platform for conversations. Symbl.ai’s real-time sentiment analysis or analytics APIs operate across protocols, channels, companies freely for developers to create freely augmented conversation experiences.

Community

Stuck? Free feel to ask us any questions on our Slack Channel or send us an email at devrelations@symbl.ai.

Real-Time Sentiment Analysis with JavaScript and Symbl.ai was originally published in JavaScript in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.