What do you understand by “Planned Actions”? Is it something required to do something according to plan? Is it something that needs to be done at a particular time? Yes, it is. “Planned Actions” is a feature present in the SUSI.AI android app that helps to schedule tasks. Yes, you heard it right! It helps to schedule the task. We can schedule some events like “setting an alarm after one minute”, “playing music after one minute”, etc. Such events require scheduling of different events at different point of time.
SUSI.AI is a smart assistant. It answers the queries that the user asks for, play music, etc. Additionally, the project also has a smart speaker part, which is very similar to Google Home. A smart speaker often has the ability to perform certain task other than answering normal queries at a scheduled time. During my GSoC period, I got the opportunity to implement such a feature of scheduling task in the SUSI.AI android app.
What is the scheduling of task or planned action?
Scheduling of task or planned actions is nothing but performing some kind of task at a scheduled time. Let’s consider “set alarm in one minute”. This means an alarm should be set that should ring up after one minute.
Let’s see how it has been implemented in SUSI.
First of all, we need to have a skill that would respond to the query asked. Consider, the skill “rocket launch”. The response for this skill is:
| { “query”: “rocket launch”, “client_id”: “aG9zdF8xNzIuNjguMTQ2LjE5MF80YTk0OGY4OA==”, “query_date”: “2019-08-19T17:31:53.670Z”, “answers”: [{ “data”: [ { “0”: “rocket launch”, “token_original”: “rocket”, “token_canonical”: “rocket”, “token_categorized”: “rocket”, “timezoneOffset”: “-330”, “language”: “en” }, { “0”: “rocket launch”, “token_original”: “rocket”, “token_canonical”: “rocket”, “token_categorized”: “rocket”, “timezoneOffset”: “-330”, “language”: “en” } ], “metadata”: {“count”: 2}, “actions”: [ { “language”: “en”, “type”: “answer”, “expression”: “starting countdown” }, { “expression”: “twelve”, “language”: “en”, “type”: “answer”, “plan_delay”: 1000, “plan_date”: “2019-08-19T17:31:54.679Z” }, { “expression”: “eleven”, “language”: “en”, “type”: “answer”, “plan_delay”: 2000, “plan_date”: “2019-08-19T17:31:55.681Z” }, { “expression”: “ten”, “language”: “en”, “type”: “answer”, “plan_delay”: 3000, “plan_date”: “2019-08-19T17:31:56.681Z” }, { “expression”: “nine”, “language”: “en”, “type”: “answer”, “plan_delay”: 4000, “plan_date”: “2019-08-19T17:31:57.682Z” }, { “expression”: “ignition sequence starts”, “language”: “en”, “type”: “answer”, “plan_delay”: 5000, “plan_date”: “2019-08-19T17:31:58.682Z” }, { “expression”: “six”, “language”: “en”, “type”: “answer”, “plan_delay”: 7000, “plan_date”: “2019-08-19T17:32:00.682Z” }, { “expression”: “five”, “language”: “en”, “type”: “answer”, “plan_delay”: 8000, “plan_date”: “2019-08-19T17:32:01.683Z” }, { “expression”: “four”, “language”: “en”, “type”: “answer”, “plan_delay”: 9000, “plan_date”: “2019-08-19T17:32:02.683Z” }, { “expression”: “three”, “language”: “en”, “type”: “answer”, “plan_delay”: 10000, “plan_date”: “2019-08-19T17:32:03.683Z” }, { “expression”: “two”, “language”: “en”, “type”: “answer”, “plan_delay”: 11000, “plan_date”: “2019-08-19T17:32:04.684Z” }, { “expression”: “one”, “language”: “en”, “type”: “answer”, “plan_delay”: 12000, “plan_date”: “2019-08-19T17:32:05.684Z” }, { “expression”: “zero”, “language”: “en”, “type”: “answer”, “plan_delay”: 13000, “plan_date”: “2019-08-19T17:32:06.684Z” }, { “expression”: “all engines running”, “language”: “en”, “type”: “answer”, “plan_delay”: 14000, “plan_date”: “2019-08-19T17:32:07.685Z” }, { “expression”: “liftoff, we have a liftoff!”, “language”: “en”, “type”: “answer”, “plan_delay”: 16000, “plan_date”: “2019-08-19T17:32:09.685Z” } ], “skills”: [“/susi_skill_data/models/general/Novelty and Humour/en/Apollo_Countdown.txt”], “persona”: {} }], “answer_date”: “2019-08-19T17:31:53.685Z”, “answer_time”: 15, “language”: “en”, “session”: {“identity”: { “type”: “host”, “name”: “172.68.146.190_4a948f88”, “anonymous”: true }} } |
The response for the above skill can be received by querying “rocket launch” at https://api.susi.ai/ or you can get it by going directly to the following link https://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=Rocket+Launch
The skill acts as a count-down timer of rocket launch. Now, we have a skill, but how are we going to implement it in android app, so that the countdown timer works properly and the output appears at the proper scheduled time. Also, we need to take care that the other processes of the app doesn’t stop or hangs up when some events are scheduled. Going through all these problems and circumstances I came up with a solution. So, let’s have a look. I made use of Handler.
Handler in Android helps to execute some tasks at a scheduled time. The tasks are executed in a different thread other than the Main UI thread. So, the other process in the app doesn’t stops and it runs smoothly.
Let’s have a look at the code:
| for (i in 0 until actionSize) { if (susiResponse.answers[0].actions[i].plan_delay.toString().isNullOrEmpty()) { planDelay = 0L } else { planDelay = susiResponse.answers[0].actions[i].plan_delay } executeTask(planDelay, susiResponse, i, date) chatView?.hideWaitingDots() } |
The code above, loops through all the actions mentioned in the skill one by one. After parsing it, the data is passed to a function executeTask(). This function performs the task to schedule the events (plan actions).
| handler = Handler() try { handler.postDelayed({ val parseSusiHelper = ParseSusiResponseHelper() parseSusiHelper.parseSusiResponse(susiResponse, i, utilModel.getString(R.string.error_occurred_try_again)) var setMessage = parseSusiHelper.answer if (parseSusiHelper.actionType == Constant.TABLE) { tableItem = parseSusiHelper.tableData } else if (parseSusiHelper.actionType == Constant.VIDEOPLAY || parseSusiHelper.actionType == Constant.AUDIOPLAY) { // Play youtube video identifier = parseSusiHelper.identifier youtubeVid.playYoutubeVid(identifier) } else if (parseSusiHelper.actionType == Constant.ANSWER && (PrefManager.checkSpeechOutputPref() && check || PrefManager.checkSpeechAlwaysPref())) { setMessage = parseSusiHelper.answer try { var speechReply = setMessage Handler().post { textToSpeech = TextToSpeech(context, TextToSpeech.OnInitListener { status -> if (status != TextToSpeech.ERROR) { val locale = textToSpeech?.language textToSpeech?.language = locale textToSpeech?.speak(speechReply, TextToSpeech.QUEUE_FLUSH, null) PrefManager.putBoolean(R.string.used_voice, true) } }) } } catch (e: Exception) { Timber.e(“Error occured while trying to start text to speech engine – ” + e) } } else if (parseSusiHelper.actionType == Constant.STOP) { setMessage = parseSusiHelper.stop removeCallBacks() chatView?.stopMic() } if (parseSusiHelper.answer == ALARM) { playRingTone() } try { databaseRepository.updateDatabase(ChatArgs( prevId = id, message = setMessage, date = DateTimeHelper.getDate(date), timeStamp = DateTimeHelper.getTime(date), actionType = parseSusiHelper.actionType, mapData = parseSusiHelper.mapData, isHavingLink = parseSusiHelper.isHavingLink, datumList = parseSusiHelper.datumList, webSearch = parseSusiHelper.webSearch, tableItem = tableItem, identifier = identifier, skillLocation = susiResponse.answers[0].skills[0] ), this) } catch (e: Exception) { Timber.e(“Error occured while updating the database – ” + e) databaseRepository.updateDatabase(ChatArgs( prevId = id, message = utilModel.getString(R.string.error_internet_connectivity), date = DateTimeHelper.date, timeStamp = DateTimeHelper.currentTime, actionType = Constant.ANSWER ), this) } }, planDelay) } catch (e: java.lang.Exception) { Timber.e(“Error while showing data – ” + e) } |
The variable planDelay determines the time after which the task needs to be executed. By default its value is 0, i.e, it gets executed at that moment only until any delay time is supplied by the skill.
In this way, planned actions have been implemented in SUSI.AI Android app.
Resources:
Blogs: Handler, Handler Implementation
SUSI Skill: Rocket Launch
SUSI.AI Android App: PlayStore GitHub
Tags:SUSI.AI Android App, Kotlin, SUSI.AI, FOSSASIA, GSoC, Android, Handler, Planned Actions
