LiveSmart AI Video feature combines the amazing capabilities of OpenAI ChatGPT with a video avatar by your choice. Interact with one of the greatest AI generated video avatars having a meaningful conversation on a variety of topics.
Requirements and Installation
PHP version
– at least PHP 8.1 and MySQL;– API keys from HeyGen or ElevenLabs (they are not mandatory) and OpenAI;
Node.js version
– at Node.js v16;– PHP and MySQL;
– API keys from HeyGen and OpenAI;
Installation is straightforward and ease. Just copy files where you need on your server and run the install folder. Follow the instructions and after the installation is done, go to Dashboard, login with admin/admin and activate your LiveSmart instance with your purchase code from CodeCanyon.
- Go to https://heygen.com/;
- Create your account;
- Go to API
- Go to https://platform.openai.com/;
- Create your account;
- Generate your APIKey
- Go to https://elevenlabs.io/;
- Create your account;
- Go to API;
- Create an API;
Video Tutorial
Usages
Combining AI avatars and ChatGPT for video or audio only interaction is incredibly resourceful. This technology provides large variety of possible scenarios and usages:
- Personal assistant for large e-commerce sites, providing information about inventory and prices;
- First line support operator;
- Healthcare operator who can check for doctor availability and schedule meetings;
- City sightseeing guide with information about coming festivals, celebrations or events;
- Travelling mate who can estimate your shortest route between two points;
- Business assistant to remind you about upcoming meetings;
- Airport information about arriving flights or helping you with booking tickets;
Advanced options allows you to define specific instructions so the avatar can intelligently capture visitor questions and give back highly accurate answers. This allows you to enormously extend the possibility of interaction with AI avatars. Following examples show how avatars can be “trained” to answer with run-time live data.
Booking Demo
You can check how a booking demo works with Angela. She will guide you on your booking tour.
Weather Forecast
You can ask Kristin for weather conditions at a specific location, for example “Tell me what is the weather like in San Francisco, CA, please”
Stock Price
This avatar, Tyler, inform you about stock price. You can ask “What is the price of crude oil”
Currency Conversion
Ask Monica to convert a currency to USD, i.e. “Convert me 100 euros”
Hotel Finder
Ask Justin for hotels near a location, i.e. “Give me hotels near London, please”
News Feed
Kayla will inform you about latest news about anything you ask here: “Latest news about Tesla”
General Queries
You can compare the above specialized avatars with Angela who is not “trained” and can not answer with run-time live data.
User Guide
Avatar Behavior
You can give some instructions to the avatar on how to respond to customer inquiries from the “Avatar attitude” field. For example: “You are a streaming avatar from LiveSmart, leading company that specialize in videos commnucations. Your name is Monica and if asked provide contact information email contact@livesmart.video. Please try your best to response with short answers, and only answer the last question. If people ask you how this works tell them this is a LiveSmart integration with HeyGen and OpenAI.”
Greeting Text
Avatar can greet the customer with some text. It can be added in the “The avatar will start the conversation with this text if provided” field. It allows adding placeholder, so it turns personally to the customer, for example: Hello {{name}}, this is Angela and I will guide you today., where {{name}} will be replaced with the name provided in the initial form. Also greeting text can be extended to add some instructions in the chat, by adding a “|” as a separator: Hello {{name}}, this is Angela. Please open the chat for more instructions|This text will appear in the chat. Then if the customer fills in John in the initial form, the avatar will greet him with “Hello John, this is Angela. Please open the chat for more instructions”, and when the customer opens the chat there will be “This text will appear in the chat”.
Advanced Options Guide
The file server/callTools.php is responsible for avatar specific live data retrieval. If you need a live weather information, you can check the included example with getCurrentWeather function in it. To specify an avatar to use it, in the dashboard from Rooms -> AI Avatars, in the Advanced tools section, in function name add getCurrentWeather, in description add Get the current weather in a given location. This is very important as it is giving instructions to AI to listen to specific keywords, for example, “give me current weather for San Francisco”. In parameters add location,unit. Unit is optional and is if you specify the temperature to be in Fahrenheit.
Please check documentation of the sample API calls.
Frequently Asked Questions
What are the limitations of using HeyGen free API key?
You can use the HeyGen free API with these limitations: 5 hours of speech per month, 15 avatars and above 120 voices. Avatar can speak up to 1000 characters of text and watermark. You can have up to three avatar sessions at once.
In order to remove this limitations, you can use the Interactive avatar plan for $100/month from here
Is OpenAI (ChatGPT) usage free of charge?
OpenAI usage is not free. They charge depending on the model you use. More info can be found in their pricing page. Tokens are chunk of text – something in-between sentences and words.
How to obtain HeyGen paid API key, credits and more avatars
You can get HeyGen credits, by purchasing them either from this form. Check for the ‘purchase API Credits’ button below. For Enterprise API Key, lowest price is $100/months.
How can I train the avatars?
1. You can add some text from the Dashboard with description of the avatar attitude and behaviour or simply with information of your live data (e-commerce site, product availability, news, events, whatever you can think of)
2. More advanced approach that requires some programming knowledge is to add your own functions and in Advanced options of the dashboard to fill in description and params. There are four type of examples I have added in the product. This example avatar is trained to give information about current weather condition at a location. If you ask the avatar “What is the weather like in San Francisco”, she will answer you with current live data for SF. Advanced options in the dashboard is like this:
Function – getCurrentWeather – this is a predefined sample function that is calling a third party site with location and unit for a current weather condition.
Description – Get the current weather in a given location – with this you are giving instructions to the avatar to listen for questions about weather and location
Parameters – location,unit – To specify location from the voice and optional unit – Celsius or Fahrenheit.
This works like this – you ask the avatar “what is the weather like in San Francisco”, avatar based on the instructions in the description understands and calls the function, the third party is returning the necessary info and avatar speaks back.
Here you can find API documentation of the sample functions. Multiple functions can be added to sequence the behavior of the avatar. For example, the booking demo is working like this: getAvailableTimeslots -> Get available timeslots and free hours -> timeslot and next function bookTimeslot -> Book a timeslot availability with email and name -> timeslot,email,name. With first set the avatar will check for availability and announce back and with the next you can book an available timeslots.
3. Last option is to have an assistant on your OpenAI dashboard. You can upload multiple files there, add descriptions and attitude to the assistant. Then this assistant can be assigned to avatar in LiveSmart Dashboard in Rooms -> AI Avatar -> Predefined Assistants.
How can I monetize from this product?
In version 1.0.6 is included SaaS and monetisation module. For more information, please refer to this article.
Can I use OpenAI gpt-4o OpenAI model?
Yes, but mind the pricing of the model.
When I click on microphone icon nothing happens (iPhone Safari and Chrome)
You need to check if you have enabled dictation on your iPhone: Go to Settings > General > Keyboard.
Turn on Enable Dictation. If a prompt appears, tap Enable Dictation. For more information check this article.
Another reason for not working microphone is if your avatar page is within an iframe and it has not the required permission. If you are adding the avatar page into another site iframe, it should look like this: <iframe id="lsv_iframe" src="YOUR_AVATAR_PAGE" allow="camera; microphone; fullscreen; autoplay; encrypted-media;" style="background-color:#ffffff; padding: 0; margin:0; border:0;" width="100%" height="600"></iframe>
Can I integrate LiveSmart AI into other platforms?
There is a WordPress plugin included in the release files. It adds LiveSmart dashboard to the WordPress admin panel, also adds shortcodes allowing you to integrate LiveSmart into your WordPress pages: [livesmart_ai_widget room="NAME_OF_THE_ROOM"].
You can easily integrate LiveSmart AI avatar page to any other platform by adding this piece of code where you need it:
<iframe id="lsv_iframe" src="YOUR_AVATAR_PAGE" allow="camera; microphone; fullscreen; autoplay; encrypted-media;" style="background-color:#ffffff; padding: 0; margin:0; border:0;" width="100%" height="600"></iframe>
0 Comments