<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Carlo Huamán on Medium]]></title>
        <description><![CDATA[Stories by Carlo Huamán on Medium]]></description>
        <link>https://medium.com/@tohure?source=rss-763dfde193e4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*H4CAuNNNgAjUjbaREp2V-Q.jpeg</url>
            <title>Stories by Carlo Huamán on Medium</title>
            <link>https://medium.com/@tohure?source=rss-763dfde193e4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 19:32:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tohure/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Android Skills & CLI en Agentes, sin humo ]]></title>
            <link>https://medium.com/@tohure/android-skills-cli-agents-sin-humo-df38d62799d0?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/df38d62799d0</guid>
            <category><![CDATA[android]]></category>
            <category><![CDATA[googleio]]></category>
            <category><![CDATA[skills]]></category>
            <category><![CDATA[mcps]]></category>
            <category><![CDATA[agents]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Wed, 27 May 2026 04:41:19 GMT</pubDate>
            <atom:updated>2026-06-02T03:27:53.823Z</atom:updated>
            <content:encoded><![CDATA[<h3>Android Skills, Android CLI, Agents CLI y más, sin humo🚬</h3><figure><img alt="Imagen portada del blog, con foto de Liam Neeson hablando de Skills" src="https://cdn-images-1.medium.com/max/1024/1*GAONeD-bqy0hJWKNYyLzdQ.png" /></figure><p>Estos meses han estado cargadas de buenas noticias para el ecosistema Android:</p><ul><li>Marzo 2026 — Se liberaron las <strong>App Functions </strong>para “<em>convertir</em>” nuestras apps en MCP → <a href="https://medium.com/@tohure/appfunctions-the-mcp-for-android-267c0afc9715">https://medium.com/@tohure/appfunctions-the-mcp-for-android-267c0afc9715</a>).</li><li>2 Abril 2026 — Google lanza <strong>Android Skills </strong>como instrucciones optimizadas para las IAs y realizar tareas específicas de Android, como migrar de XML a Compose o configurar el SDK → <a href="https://developer.android.com/tools/agents/android-skills">https://developer.android.com/tools/agents/android-skills</a>.</li><li>16 Abril 2026 — Google anuncia <strong>Android CLI, </strong>una interfaz de comandos diseñada para que agentes de IA gestionen el ciclo de vida de desarrollo en Android mediante comunicación estructurada y optimización de tokens → <a href="https://android-developers.googleblog.com/2026/04/build-android-apps-3x-faster-using-any-agent.html">https://android-developers.googleblog.com/2026/04/build-android-apps-3x-faster-using-any-agent.html</a>.</li><li>19 Mayo 2026 — En el Google I/O de este año se ha mencionado aún más mejoras para las Skills, Android CLI, la migración de gemini-cli a antigrativy-cli y cómo todo se integra en el ecosistema con herramientas como Antigrativy o Android Studio.</li></ul><p><a href="https://android-developers.googleblog.com/2026/05/17-things-android-developers-google-io.html">17 Things to know for Android developers at Google I/O</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*JS38iDoW9wXlqKk9dMRjVw.gif" /></figure><h3>Primer Capítulo: Las Android Skills</h3><h4>¿Qué son las skills? (a secas)</h4><p>No son otra cosa que reglas, comentarios, documentación ordenada para que tu LLM, Agente, IA, Chatbot de preferencia, entienda y expanda más su conocimiento sobre una tarea específica que quieres que haga en un documento <strong>markdown </strong>para facilitar su lectura.</p><p>Entonces si son reglas o pasos a seguir, ¿las puedo escribir como quiera?. Si puedes. Pero lo bueno de las de Google es que siguen el <strong><em>estándar abierto</em></strong> para skills (<a href="https://agentskills.io/home">agentskills.io</a>), lo que facilita que cualquier herramienta la pueda leer correctamente (y no sólo Gemini o Claude o GPT o Copilot).</p><p>Un skill se vería mas o menos así:</p><pre>---<br>name: nombre_de_tu_skill<br>description: Instrucciones de lo que debe hacer o no tu skill y<br>que es lo que necesita para conseguirlo<br>license: terminos y condiciones de tu skill<br>metadata:<br>  author: Autor<br>  last-updated: &#39;2026-05-20&#39;<br>  keywords:<br>  - android<br>  - ui<br>  - compose<br>---<br><br>## Descripcion1<br><br>La app debe hacer:<br><br>- Uso de Compose para todas las pantallas.<br>- Uso de Jetpack Navigation.<br><br>## Descripcion2<br><br>Para hacer otra cosa el app debe:<br><br>- Paso 1: Hacer algo<br>- Paso 2: Hacer algo más<br>- Paso 3: Hacer algo mucho más (para más información click aqui)<br></pre><p>En este documento va toda la descripción que necesites, y al final el archivo debe llamarse SKILL.md y para diferenciarlo de otras lo debes poner en una carpeta con un nombre descriptivo.</p><p>Si tu skill crece demasiado, siempre puedes usar una estructura de carpetas para, dentro de tu misma skill, referenciar a más archivos que ayuden a entenderla mejor, sólo la principal debe llamarse SKILL.md y los demás documentos colocarlos en una carpeta como <strong><em>reference</em></strong>. Al final tendrás una estructura de carpetas algo así:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/336/1*m7Z0Yz9TuNZL5eSBwsLmOw.png" /><figcaption>Estructura de carpetas de una Skill Android | Fuente: Github Android</figcaption></figure><h4>Y las ¿Android Skill?</h4><p>Con el mundo de las skills creciendo, Google y el equipo de Android recopilaron las buenas prácticas así cómo herramientas y documentación de trabajo para el desarrollo en Android para crear sus propias skills.</p><p>Pero… ¿Por que haríamos algo así? ¿Para que tener la información tan organizada?</p><p>Y esto es porque, cuando trabajamos con Agentes, LLMs, etc; no podemos darnos el lujo de gastar tanto tiempo o tokens escribiendo:</p><blockquote>Por favor, hazlo con buenas prácticas de tal tal tal y guíate de esta documentación y este link y este otro link, etc. Luego cuando hagas eso haz aquello con esta otra documentación según este link y estas buenas prácticas, porfis!</blockquote><p>Así que en lugar de tener tantos modismos y <em>porfis </em>en tu prompt, Google escribió una serie de <strong><em>Android Skills </em></strong>que se pueden traducir en <strong>prompts más ligeros</strong> y <strong>agentes más inteligentes.</strong><em> (Y menos tokens gastados en tu sesión actual, y lo sabes :v ).</em></p><p><a href="https://developer.android.com/tools/agents/android-skills">Overview of Android skills | Android Studio | Android Developers</a></p><p>Las cuales serán muy importantes para el siguiente capítulo.</p><h3>Segundo Capítulo: Android CLI</h3><blockquote>CLI o <em>Command Line Interface</em> o, para los patas, <strong><em>Interfaz de Línea de Comandos, </em></strong>que viene a ser cualquier programa con el que se interactúa escribiendo comandos de texto en lugar de hacer click en iconos o botones.</blockquote><p>¿Por qué necesitamos una CLI?, ¿no son mejores los programas gráficos a colores y muchos botones?, si pero aunque no lo crean, a un humano, mandar señales del cerebro a la mano para moverla y esta mueva a su vez un mouse con todo y cursor hasta cierto botón de la pantalla, toma tiempo, y aunque sean milisegundos por ser un pro del Age of Empires II o StarCraft o CounterStrike, la computadora lo hace mucho más rápido con ayuda de las CLI.</p><figure><img alt="Gif de aldeanos del juego Age of Empires construyendo una casa" src="https://cdn-images-1.medium.com/max/250/1*ORLz4evxRRL4SPl6tdpO3A.gif" /><figcaption>Mi APM en juegos es tan bajo que sí o sí necesito una CLI</figcaption></figure><p>Entonces Android CLI viene a ser el conjunto de herramienta que siempre usamos para desarrollar en Android pero por línea de comandos.</p><p>Pero si un hater dice: siiii pero puedes correr ya ADB desde consola y lo mismo con gradle porque ambos ya son sus propias CLI, ¡no necesitas otra herramienta más!. Tu puedes decirle: La gran diferencia es que <strong><em>Android CLI</em></strong> actúa como un mánager automatizado. Ya no eres tú el que tiene que ir a la web, descargar un ZIP de Platform-Tools, configurar el PATH de adb, instalar JDK, o pelear con versiones de Gradle. La CLI se encarga de orquestar, descargar y ejecutar todo lo que haga falta en segundo plano (Cómo las Skills que dijimos en el capítulo anterior).</p><p>Por debajo, los binarios y compiladores de Android (como las plataformas del SDK o las tecnologías de emulación) siguen existiendo, porque son físicamente necesarios para que el código se convierta en una app funcional, pero ahora es la CLI quien lo va a llamar y gestionar.</p><p>Para descargar la CLI pueden usar el siguiente enlace:</p><p><a href="https://developer.android.com/tools/agents/android-cli/download">Download Android CLI | Android Studio | Android Developers</a></p><p>Una vez instalado en su máquina, sólo se invoca.</p><figure><img alt="gif mostrando una terminal corriendo comandos como docs o skills de la android-cli" src="https://cdn-images-1.medium.com/max/870/1*tVNw1NWyfPF_6QizS2r8eg.gif" /><figcaption>Terminal corriendo comandos en Android CLI como docs y skill</figcaption></figure><p>Con esta CLI facilitamos el uso de emuladores, creación de proyectos, estructuras, búsqueda de documentación, etc, y todo apoyado por LAS SKILLS, de esa forma la CLI se ayuda a sí misma a hacer todo más rápido.</p><p>Pero, ¿igual crearíamos un proyecto así desde cero?, ¿Así se pondrían a buscar documentación? ¿no es mejor desde su buscador o IA favorita? (¿A que no?)</p><p>Para estos casos es mejor usar lenguaje natural para hacer los proyectos si no, para eso sigo usando un programa gráfico y puros clicks a lo loco. Entonces, ¿si Android CLI sólo acepta comando android, cómo lo conecto a una IA?.</p><h3>Tercer Capítulo: Conectando Android CLI a otros CLI’s (o tu Agente CLI favorito)</h3><p>Cómo dije en el capítulo anterior, Android CLI es genial, pero siendo sinceros, en plena era de la IA, es preferible usar lenguaje natural para comenzar a crear un proyecto en Android. ¿Cómo?. Conectando Android CLI a cualquier Agente de IA que usemos donde:</p><ul><li>Escribiremos en nuestro <strong><em>Agente</em></strong> (Gemini, Claude, Codex, Copilot) a través de su propio CLI, una petición en <strong><em>lenguaje natural.</em></strong></li><li>El <strong>agente</strong> recibirá la petición del usuario y la transformará en una orden entendible (comando) que pueda ejecutar a través del <strong><em>Android CLI</em></strong>.</li><li><strong><em>Android CLI</em></strong> recibirá la instrucción pura (sin palabras innecesarias, sólo comandos), para luego llamar a los CLIs necesarios (cómo adb o gradlew).</li><li>Finalmente se ejecutará la tarea.</li></ul><pre>[ Nivel 0: CLI de Intención (Lenguaje Natural) ]<br>         ┌─────────────────────────┐<br>         │   `claude` / `gemini`   │  &lt;-- Tú le hablas en cristiano: <br>         └────────────┬────────────┘      &quot;Corre la app en el emulador&quot;<br>                      │<br>                      ▼ (Traduce la intención a comandos estructurados)<br>                      <br>[ Nivel 1: CLI Orquestadora (Herramientas unificadas) ]<br>         ┌─────────────────────────┐<br>         │      android cli        │  &lt;-- Entiende el ecosistema Android:<br>         └────────────┬────────────┘      `android run --device=emulator`<br>                      │<br>       ┌──────────────┴──────────────┐<br>       ▼ (Automatiza la ejecución)   ▼ (Delega la compilación)<br>       <br>[ Nivel 2: CLIs Especializadas (Bajo Nivel) ]<br>┌─────────────────────────────┐   ┌─────────────────────────────┐<br>│             `adb`           │   │          `gradlew`          │ &lt;-- Hacen el<br>└─────────────────────────────┘   └─────────────────────────────┘     trabajo<br> (Habla con el dispositivo)        (Compila el código a binario)      sucio.</pre><p>Para poder decirle a Android CLI que sea visible a todos los agentes que estemos usando, debemos usar el comando android init, lo cual creará <strong><em>skills</em> </strong>específicas de uso para las otras CLI’s de cómo Android CLI.</p><figure><img alt="Imagen de una consola corriendo el comando “android init”" src="https://cdn-images-1.medium.com/max/1024/1*t0pnvhC6KRk9zQWLtXygCw.png" /><figcaption>Android CLI instalándose para la mayoría de agentes de IA</figcaption></figure><p>Ya con ello podemos usar tanto las skills como el cli de android en nuestros agentes.</p><h4>Gemini CLI</h4><p>Ya no intenté seguir la prueba en Gemini CLI porque esta siendo migrada a Antigravity CLI</p><figure><img alt="Imagen de una consola mostrando gemini-cli, con un mensaje de que va a ser reemplazado por antigravity-cli" src="https://cdn-images-1.medium.com/max/970/1*1a85ZSBZmDUxSeg4QGGDMQ.png" /><figcaption>Mensaje de Gemini CLI diciendo que pronto partira :(</figcaption></figure><p><a href="https://developers.googleblog.com/an-important-update-transitioning-gemini-cli-to-antigravity-cli/">An important update: Transitioning Gemini CLI to Antigravity CLI</a></p><h4>Antigravity CLI</h4><p>En el paso de instalación de instalación de android-cli vimos como estaba instalando las skills explícitamente para poder ser usada por el agente de antigravity (y muchos otros), por lo que ahora cuando escribimos un prompt en <strong><em>lenguaje natural, </em></strong>nuestro agente (Antigravity) detecta gracias a la metadata dada por los skills que comandos debe llamar y así evita búsquedas innecesarias de información o gastar más tokens en procesos más largos (como un gradle con todos los parámetros necesarios)</p><figure><img alt="Consola con Antigravity CLI llamando comandos para crear un proyecto Android con su propia cli." src="https://cdn-images-1.medium.com/max/976/1*SI-M4QiS0j5qnJw-_5ZKKw.png" /><figcaption>Consola con Antigravity CLI llamando comandos para crear un proyecto Android con su propia cli.</figcaption></figure><p>Esto no significa que el agente no pueda hacerlo por si mismo, sólo que sin la información de skills o clis complementarias, tendría primero que buscar cómo funciona las clis de gradle, adb, java, etc, luego pasar parámetros de cada una, recibir las respuestas, procesarlas, etc, todo ello implica más gastos de tokens. Ahora, por las android skills sabe que tiene que llamar por ejemplo android create y no preocuparse por nada más, porque el comando hace todo el trabajo.</p><figure><img alt="Consola con Antigravity CLI con los comandos llamados para crear una aplicación de Android." src="https://cdn-images-1.medium.com/max/975/1*yEeaog6xl-yd20jsz3VK6A.png" /><figcaption>Consola con Antigravity CLI con los comandos llamados para crear una aplicación de Android.</figcaption></figure><h4>Claude Code</h4><p>Y así como con Antigravity, con cualquier agente pasa lo mismo, en la siguiente imagen pueden ver como Claude está usando las skills y cli de Android para procesar la respuesta.</p><figure><img alt="Consola con Claude Code corriendo un comando de Android CLI" src="https://cdn-images-1.medium.com/max/767/1*o8VjQwRkkmujYp_IwLjvRA.png" /><figcaption>Consola con Claude Code corriendo un comando de Android CLI</figcaption></figure><h3>Conclusiones</h3><ul><li>Definitivamente las Skills de Android y su CLI, potencian un 200% cualquier agente en temas de desarrollo de Android o Kotlin (porque recuerden hay CLI’s y Skills para todos los gustos), sea cual sea el agente al cual estés acostumbrado.</li><li>No existe algo como “las skills definitivas”, tu mismo puedes crear unas que se acomoden a tu entorno o modo de trabajo. Sólo procura seguir el estándar abierto para que sean compatibles con cualquier agente.</li><li>Es bueno conocer que existe más de una herramienta que ayude a nuestro desarrollo, pero más aún, saber cómo funciona para así poder sacarle el máximo provecho.</li></ul><p>Si tienes dudas, escríbeme un DM a Twitter (aka X). Gustoso de ayudar.</p><figure><img alt="Imagen de logo android" src="https://cdn-images-1.medium.com/max/568/1*AZ6j2Q_aHMwcm49cQX5wOA.png" /><figcaption>¡¡Hasta la próxima!!</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df38d62799d0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AppFunctions: Convierte tu Android App en una herramienta para Agentes (MCP)]]></title>
            <link>https://medium.com/@tohure/appfunctions-the-mcp-for-android-267c0afc9715?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/267c0afc9715</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[mcps]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[mcp-protocol]]></category>
            <category><![CDATA[androiddev]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Fri, 17 Apr 2026 05:10:40 GMT</pubDate>
            <atom:updated>2026-04-17T05:10:40.549Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6XX_odAZEAOlBuZlB77lcQ.jpeg" /></figure><p>El día de hoy quiero contarles una novedad que ha traído Google para las Android App, las AppFunctions, pero como toda buena historia, todo tiene un principio.</p><h3><strong>Capítulo 1: Una mirada al pasado — Las App Actions</strong></h3><p>Recuerdo allá por el 2020 cuando Google Assistant aún vivía (Qué tiempos aquellos 🥺), qué Google lanzó para Android una característica llamada <strong>App Actions, </strong>la cual me permitía conectar mi Android App con Google Assistant a través de comandos de voz (Si, para aquel entonces para era mágico 🪄).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BLNHTfGb_r3GlxEl.png" /><figcaption>Fuente: Google Developers | Flujo de consulta de usuario de una App Action.</figcaption></figure><p>Básicamente, las <strong>App Actions </strong>funcionaban gracias a que <strong>Google Assistant</strong> escuchaba una <strong>petición del usuario</strong> (<em>query),</em> y lo modelaba a un nivel <em>legible</em> para la aplicación (<em>Built-in Intents, </em><strong><em>BII</em></strong>). Para que tu aplicación fuere capaz de <em>entender</em> y cumplir este BII (<em>fulfillment)</em>, tenía que tenerla registrada en un archivo dentro de tu Android App llamado <strong>shortcuts.xml.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/249/1*boBcQMJWuNQYMcadbnOVeQ.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oiMfOd1LSYXbQWKV1mEv2g.png" /><figcaption>App Actions Adidas Run (izquierda) | Assistant Widgets (derecha) | Fuente: Google I/O 21</figcaption></figure><p>Podías iniciar con tu voz una característica de tu aplicación favorita, o en todo caso, no era necesario abrir toda la aplicación si no revisar tu última acción hecha dentro del mismo Google Assistant, sin abrir tu app entera o buscarla en un mar de aplicaciones instaladas.</p><p>Pero al parecer no despegó tanto como se pensó…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_qXphQWurcvSHtT2C0gTNQ.png" /></figure><p>Estamos en 2026, tuvimos una pandemia, el hombre volvió a la luna con el proyecto Artemis II, Perú tuvo unos 8 presidentes desde el 2020 y el boom de la IA se da en todo el mundo: Gemini, Claude, Grok, Agentes, RAG, MCP, etc. Empezó la gran era de la IA. (Poner aquí opening: We Are! 🏴‍☠️⚓️ | Si entendiste eres de los míos)</p><h3>Capítulo 2: Agentes, Tools y MCPs</h3><p>No pretendo que este articulo sea muy largo, pero podemos hacer una breve mención a que son los Agentes, Tools y los MCPs (claro, todo en el contexto de la IA).</p><ul><li><strong>Agentes:</strong> Sistemas basados en LLMs configurados con un bucle de razonamiento autónomo. A diferencia de un chatbot estándar, un agente descompone una solicitud compleja en subtareas lógicas, determina qué información le falta y decide qué funciones externas debe invocar para alcanzar un estado final u objetivo.</li><li><strong>Tools:</strong> Son interfaces de programación (APIs) expuestas al modelo mediante esquemas descriptivos, generalmente en formato <strong>JSON Schema</strong>. Cada <em>tool</em> define un nombre, una descripción técnica de su propósito y una estructura estricta de parámetros de entrada. El modelo no ejecuta el código, sino que genera un “tool call” con los argumentos necesarios para que el sistema local realice la ejecución.</li><li><strong>Model Context Protocol (MCP):</strong> Estándar abierto y agnóstico, introducido por <em>Anthropic (los de Claude)</em>, diseñado para resolver la fragmentación entre los modelos de lenguaje (LLMs) y las fuentes de datos o herramientas externas. Actúa como una capa de abstracción que permite a las aplicaciones de IA conectarse a repositorios de datos, APIs y herramientas locales mediante una interfaz unificada, eliminando la necesidad de escribir integraciones personalizadas para cada modelo o plataforma.</li></ul><p>Cuando un usuario solicita una tarea compleja, el <strong>Agente</strong> analiza el <em>propósito final </em>de la petición del usuario y utiliza el protocolo <strong>MCP</strong> para identificar qué capacidades externas <em>tiene a su disposición</em> de forma estandarizada. Una vez localizadas, el agente solicita la ejecución de una <strong>Tool</strong> específica enviando los parámetros necesarios a través del protocolo, el cual procesa la acción técnica y retorna el resultado; esto permite que el agente incorpore datos o acciones reales en su razonamiento para completar la tarea de forma exitosa.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/1*w7uiw3sa2AhVWMg3XOMoGA.png" /><figcaption>Protocolo MCP (Agents &amp; Tools)</figcaption></figure><p>Google ha implementado su propia versión de este concepto directamente en el núcleo de Android. Tu app se convierte en un “Server de herramientas (Tools)” local, y Gemini (o cualquier agente autorizado) es el cliente que las consume.</p><h3>Capítulo 3: Llevando MCP a Android, AppFunctions</h3><p>Google lanzó una característica recientemente para los móviles: Las <strong>AppFunctions. </strong>Una nueva característica de Android diseñada para que las aplicaciones <strong>compartan funciones específicas directamente con el sistema operativo</strong> y con diversos agentes de inteligencia artificial (como Gemini 😉).</p><p>Mientras que <strong>MCP</strong> (Model Context Protocol) conecta agentes con herramientas en servidores, las <strong>AppFunctions</strong> llevan este mismo mecanismo a Android.</p><h4>¿Cómo funcionan?</h4><p>Si tuviéramos que resumirlas en un paso a paso sería de la siguiente manera:</p><ul><li><strong>Definición de la función:</strong> El desarrollador identifica una <strong><em>funcionalidad</em></strong> específica de su app (“crear una nota” o “enviar un mensaje”) y la marca utilizando anotaciones (@AppFunction) de la biblioteca de Jetpack para AppFunctions. Esto permite que el compilador genere los metadatos necesarios.</li><li><strong>Descripción para la IA:</strong> Para cada función, se proporciona una descripción en lenguaje natural por medio de la documentación debida de la función (Kdoc). Esto es crucial porque ayuda a los LLMs a entender <em>cuándo</em> y <em>para qué</em> deben usar esa herramienta específica basándose en lo que el usuario pida.</li><li><strong>Implementación del Servicio:</strong> Se implementa una clase que extiende de AppFunctionService. Este servicio actúa como &quot;puente&quot; que el sistema llamará cuando un agente de IA decida que necesita ejecutar esa función.</li><li><strong>Registro en Manifest:</strong> La aplicación debe declarar este servicio en su AndroidManifest.xml, con el permiso BIND_APP_FUNCTION_SERVICE. Esto garantiza que solo el sistema operativo Android pueda conectarse directamente al servicio de la app.</li><li><strong>Descubrimiento (Discovery):</strong> Cuando un usuario hace una solicitud (por ejemplo: “Gemini, agenda una reunión para mañana”), el sistema o el asistente utiliza el AppFunctionManager para buscar entre todas las apps instaladas aquellas que tengan una &quot;AppFunction&quot; que coincida con esa intención.</li><li><strong>Verificación de Permisos:</strong> Antes de ejecutar cualquier cosa, el sistema verifica que el “llamador” (el asistente o la app que intenta usar la función) tenga el permiso EXECUTE_APP_FUNCTIONS. Esto asegura que solo agentes autorizados interactúen con tu app.</li><li><strong>¡Ejecución!:</strong> Una vez identificada la función correcta, el sistema invoca el método de ejecución. La app procesa los datos recibidos (como el título o la hora de la reunión extraídos por la IA), realiza la tarea y devuelve un resultado al asistente para que este confirme al usuario que la tarea se completó.</li><li><strong>Requisitos del Sistema:</strong> Todo este proceso ocurre de forma local en el dispositivo y requiere <strong>Android 16 o superior</strong> para funcionar.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9usA67W-DuBkwdR_Xihh9g.png" /><figcaption>Flujo de cómo un Agente expone y ejecuta las AppFunctions | Fuente: Android Developers Documentation</figcaption></figure><h3>Capítulo 4: Conclusiones — ¿Qué sigue ahora?</h3><p>Históricamente, los usuarios interactúan con las apps a través de interfaces visuales (UI). Con la llegada de los agentes de IA (como Gemini), el paradigma cambia.</p><blockquote>La IA no quiere “ver” tu app; quiere “usar” sus capacidades. Aquí es donde entran las <strong>AppFunctions</strong>.</blockquote><h4>¿Casos de Uso?</h4><p>No sé, se me ocurren:</p><ul><li><strong>Productividad:</strong> “Crear tarea”, “Buscar factura”.</li><li><strong>Media:</strong> “Reproducir playlist de Jazz”, “Editar brillo de foto”. (Muy Google Photos)</li><li><strong>Social:</strong> “Enviar mensaje a X grupo”. (¿Tu sistema de mensajería?)</li></ul><h3>Mensaje final: Prepárate para la “Era Agéntica”</h3><p>Si tu app no expone AppFunctions, en el futuro cercano será invisible para los asistentes/agentes de IA. Convertir tu app en un “Toolbox” no es opcional si quieres seguir siendo relevante en Android 16. ¯\_(ツ)_/¯</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/0*FHuLJxI1sFBE0wMA.jpg" /><figcaption>¡¡Caminen al futuro!!</figcaption></figure><h3>Referencias</h3><ul><li><a href="https://developer.android.com/ai/appfunctions">Overview of AppFunctions | AI | Android Developers</a></li><li><a href="https://github.com/tohure/CapabilitiesShortcuts">GitHub - tohure/CapabilitiesShortcuts: Proyecto que implementa Android Shortcuts &amp; Capabilities API</a></li><li><a href="https://github.com/tohure/AssistantShortcuts">GitHub - tohure/AssistantShortcuts: How to integrate classic App Shortcuts with Google Assistant through the new Capabilities API</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*sz5IepfVrd84yqvTat3Cig.png" /><figcaption>¿Será?🤔</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=267c0afc9715" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Blindaje de WebViews: Implementando Channel Messaging API para una comunicación Android-JS libre de…]]></title>
            <link>https://medium.com/@tohure/blindaje-de-webviews-implementando-channel-messaging-api-para-una-comunicaci%C3%B3n-android-js-libre-de-9c2bacdb6e17?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c2bacdb6e17</guid>
            <category><![CDATA[android]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[mobile-app-development]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Fri, 30 Jan 2026 23:09:13 GMT</pubDate>
            <atom:updated>2026-01-30T23:09:13.719Z</atom:updated>
            <content:encoded><![CDATA[<h3>Blindaje de WebViews: Implementando Channel Messaging API para una comunicación Android-JS libre de inyecciones</h3><figure><img alt="Dos muñequitos de android comunicandose por un cable." src="https://cdn-images-1.medium.com/max/1024/1*UC5iYTHKCa5yUi2GdNlB9Q.jpeg" /></figure><p>Les ha pasado que les han pedido poner, renderizar, llamar, etc; ¿una web dentro de su aplicación mobile?. Y luego les piden algo cómo: ey, la web tiene un dato que necesito enviarle a la aplicación mobile y luego que la aplicación mobile le envíe otro dato a esta misma web, ¿Se puede?</p><figure><img alt="Meme de Timmy Turner preguntando: ¿Qué podría salir mal?" src="https://cdn-images-1.medium.com/max/624/1*ukdkbyaQ5S8xbW1qCw9_0w.png" /></figure><p>Tuve ese caso particular en mi día a día donde trabajaba con un Android Webview, una Web con MicroFrontends, y mucha procastinación. En algún punto me tocaba realizar la susodicha comunicación. ¿Qué hacer en estos casos?, pues su respectivojavaScriptEnabled y addJavaScriptInterface , ¿Qué podría salir mal?</p><figure><img alt="Imagen de Android Studio indicando un warning acerca de que es mlao usar javaScriptEnabled" src="https://cdn-images-1.medium.com/max/509/1*N6UBksXW7kPAA_FQhWBhjw.png" /></figure><p>Cómo bien ya vieron, el mismo Android Studio nos dice que existe la posibilidad de generarle vulnerabilidades de tipo <strong>XSS</strong> a nuestra aplicación.</p><blockquote><strong>XSS o Cross-Site Scripting</strong>, es una vulnerabilidad que permite a atacantes inyectar scripts maliciosos (como JavaScript) en sitios web confiables.</blockquote><blockquote>Mas info aquí 👉🏽 <a href="https://owasp.org/www-community/attacks/xss/">owasp.org/www-community/attacks/xss/</a></blockquote><p>Si bien este ha sido el método preferido por mucho tiempo, cómo pudieron observar (o leer) es <strong>riesgoso</strong>, pues expone que nuestra página web pueda ser vulnerada así cómo también nuestra aplicación. Entonces, ¿Cómo puedo seguir la comunicación de estas dos partes?, existe otra manera más segura de comunicarte con tu web llamada <strong>Channel Messaging.</strong></p><blockquote>La <strong>API de Channel Messaging</strong> permite que dos partes de un sitio web como la página principal y un IFrame se comuniquen directamente entre sí a través de un “túnel” <strong>privado</strong> (<em>importante lo de privado</em>) con dos extremos, donde cada parte recibe un <strong>puerto</strong> para enviar y recibir mensajes de forma bidireccional y segura.</blockquote><blockquote>Mas info aquí 👉🏽 <a href="https://developer.mozilla.org/en-US/docs/Web/API/Channel_Messaging_API">developer.mozilla.org/en-US/docs/Web/API/Channel_Messaging_API</a></blockquote><h4><strong>Channel Messaging (</strong>Secure Communication)</h4><p><strong>Channel Messaging </strong>trabaja para comunicar distintas partes de la web, pero en esta ocasión, lo haremos para comunicar la Web (HTML) con nuestro Webview (Android) de forma segura con ayuda de un método propio de WebView llamado <strong>createWebMessageChannel(). </strong>Veamos cómo funciona:</p><ol><li>Usamos la función <strong>createWebMessageChannel() </strong>en Android<strong> </strong>o <strong>new MessageChannel() </strong>en Javascript, para mandar a crear <strong>dos puertos (MessagePort</strong>).</li><li>El motor de JavaScript V8 (Chromium), crea estos dos puertos que son <strong><em>Transferable Objects</em></strong> en C++, objetos <strong>transferibles, </strong>mas no copiables.</li><li>El motor V8, crea un túnel binario, haciendo que la comunicación viaje por el bus de mensajes interno de <strong>Chromium</strong>. Esta comunicación ocurre a nivel de procesos del sistema operativo o hilos de memoria compartida en C++, por lo que nadie más puede “escuchar” ese canal.</li><li>Luego, nosotros <strong>transferimos uno de esos puertos</strong> para empezar a comunicarnos. (Web → IFrame/Worker) o (Nativo → Web)</li><li>El motor V8 le dice al motor de C++: <em>“El contexto A ya no es dueño de este puerto, ahora le pertenece al contexto B” </em>con ello<em>, l</em>a transferencia de nuestro puerto ha sido realizada.</li><li>Finalmente, ya podemos comenzar a transferir información de forma “segura” entre nuestra App y la Web. (Y digo segura entre comillas porque aún hay otros matices por revisar).</li></ol><p>Importante señalar por si un hater me dice que igual estoy simulando “el puerto” en Android por que no es HTML/Javascript:</p><blockquote>En Android, el WebMessagePort es un <strong>proxy nativo</strong> de un objeto MessagePort de C++. No hay simulación, es una conexión directa al motor de mensajería de Chromium y esto pues porque <strong>Android WebView ES un navegador (Chromium)</strong>.</blockquote><blockquote>Más información aquí 👉🏽 <a href="https://developer.android.com/reference/android/webkit/WebMessagePort">developer.android.com/reference/android/webkit/WebMessagePort</a></blockquote><blockquote>En resumen: el uso de puertos es lo mismo que tener 2 walkie talkies, uno te lo quedas tu, y el otro se lo dás a la web, sincronizan una frecuencia secreta y ya. Nadie más los escuchará.</blockquote><figure><img alt="Imagen de dos walkie talkies comunicandose" src="https://cdn-images-1.medium.com/max/785/1*POfvXKw1pBtNApCorJ3cwA.png" /></figure><p>Tenemos el caso de uso, tenemos una propuesta, tenemos el cómo funcionaría la propuesta, sólo nos falta la demo. Por que sí, puro bla bla bla, puro floro y sin demo, sólo es humo 🏃🏽💨🏃🏾‍♀️ ¯\_(ツ)_/¯</p><h3>El proyecto</h3><p>El proyecto tiene 3 partes:</p><ul><li>Configuración del cliente (Webview)</li><li>Configuración de la lógica de Puertos, envío de mensajes y validación de origen</li><li>Página HTML con validación de puertos y su respectiva lavada de cara por mano de la buena IA. (Sorry, el CSS nunca fue lo mío)</li></ul><blockquote>El proyecto puede enviar mensajes a la web (y esta los pinta en una lista) y la web puede enviar mensajes a la app mobile (y mostrarlos como un Toast).</blockquote><h4>Configuración del cliente (Webview)</h4><p>En pleno 2026 sería raro usar XML, así que el Webview lo pintaré en un @Composable con ayuda de un <a href="https://developer.android.com/develop/ui/compose/migrate/interoperability-apis/views-in-compose">AndroidView</a> y su factory.</p><pre>AndroidView(<br>  factory = {<br>    WebView(context).apply { . . . },<br>    update = { webviewInstance }<br>)    </pre><p>Una vez que tenga la instancia del webview en el update del factory, la puedo referenciar a una variable para así llamar al webview en algún otro punto de ser necesario.</p><p>En la configuración del webview (dentro del factory) existen algunos puntos importantes a revisar:</p><ul><li>Configurar el console log de la web para poder revisar en su Logcat de Android, todo lo que pasa por detrás.</li></ul><pre> webChromeClient = object : WebChromeClient() {<br>    . . .<br><br>    override fun onConsoleMessage(consoleMessage: ConsoleMessage?): Boolean {<br>      Log.d(TAG,<br>            &quot;${consoleMessage?.message()}<br>             ${consoleMessage?.lineNumber()}<br>             ${consoleMessage?.sourceId()}&quot;)<br>      return true<br>    }<br>   . . . <br>}</pre><ul><li>Esperar que la página termine de cargar para poder crear los puertos, esto lo hacemos con el WebViewClient</li></ul><pre>webViewClient = object : WebViewClient() {<br>  override fun onPageFinished(view: WebView?, url: String?) {<br>      super.onPageFinished(view, url)<br>      view?.let { webViewModel.createMessageChannel(it) }<br>  }<br>}</pre><ul><li>Llamar a nuestra página/demo (archivo local) con loadDataWithBaseURL().</li></ul><pre>val BASE_URL = &quot;https://app.tohure.assets&quot;<br>val html = context.assets.open(&quot;index.html&quot;).bufferedReader().use(BufferedReader::readText)<br>loadDataWithBaseURL(BASE_URL, html, &quot;text/html&quot;, &quot;UTF-8&quot;, null)</pre><p>Aquí si me gustaría detenerme un segundo, porque sé que en muchos tutoriales encontrarán que el html se puede cargar de forma más rápida con un simple:</p><pre>webView.loadUrl(&quot;file:///android_asset/index.html&quot;)</pre><p>Entonces, ¿porque me complique con un loadDataWithBaseURL()?</p><p>Más adelante hablaré de ello, pero por ahora sólo les adelantaré que cuando cargan un <strong>archivo local </strong>con file:///, el WebView le asigna un <strong>origen</strong> <strong>null</strong> por razones de seguridad. Y un origen <strong>null</strong> no nos ayuda para ejemplificar la seguridad que deba tener un puerto.</p><p>Con loadDataWithBaseURL() puedo “asignar” una URL fake que me ayude a simular este <strong>origin</strong> que necesitaré más adelante. (Sólo tengan cuidado de no usar alguna URL real de por ahí xD)</p><h4>Configuración de los puertos</h4><ul><li>Primero necesitamos crear los puertos para la comunicación, para ello usaremos createWebMessageChannel() y los almacenaremos en dos <a href="https://developer.android.com/reference/android/webkit/WebMessagePort">WebMessagePort</a> (el <a href="https://developer.mozilla.org/en-US/docs/Web/API/MessagePort">MessagePort</a> que mencionamos al inicio💡).</li></ul><pre>private var port1: WebMessagePort? = null<br>private var port2: WebMessagePort? = null<br><br>fun createMessageChannel(webView: WebView): Array&lt;WebMessagePort&gt; {<br>  val channel = webView.createWebMessageChannel()<br>  port1 = channel[0]<br>  port2 = channel[1]<br>  return channel<br>}</pre><p>El puerto 1 será el nuestro (de la app) el que usaremos para escuchar/enviar mensajes, y el puerto 2 será el que enviaremos a la web para que pueda comunicarse con nosotros.</p><ul><li>Procedemos a configurar el <strong>Puerto 1</strong> para que escuche los mensajes entrantes. (Mensajes que serán enviados desde el Puerto 2). Llamaremos a la función <a href="https://developer.android.com/reference/android/webkit/WebMessagePort#setWebMessageCallback(android.webkit.WebMessagePort.WebMessageCallback,%20android.os.Handler)">setWebMessageCallback()</a> que es parte de WebMessagePort, donde llamaremos a la clase <strong>WebMessageCallback </strong>que trabaja como<strong> listener</strong> para manejar el evento onMessage de <strong>MessagePort</strong>. Donde nos llegarán los mensajes recibidos por este puerto.</li></ul><pre>port1?.setWebMessageCallback(object : WebMessagePort.WebMessageCallback() {<br>    override fun onMessage(port: WebMessagePort?, message: WebMessage?) {<br>      super.onMessage(port, message)<br><br>      val response = message?.data<br><br>      webViewState.update {<br>        it.copy( response )<br>      }<br>    }<br>})</pre><blockquote>IMPORTANTE: el callback procesa los mensaje en el <strong>Main Thread</strong>, por lo que la carga de información debe ser manejada con cuidado, y de ser el caso de mucho procesamiento, pasarlo a un hilo diferente.</blockquote><ul><li>El <strong>puerto 2, </strong>es el que enviaremos a la web para que sepa por donde comunicarse con nosotros. Para poder enviarlo, haremos uso de <strong>postWebMessage(), </strong>que me permite enviar un mensaje a la web, pero al mismo tiempo, me permite enviar un array de puertos que quiero que la web conozca.</li></ul><pre>val webMessage = WebMessage(&quot;message_to_web&quot;, arrayOf(port2))<br>webView?.postWebMessage(webMessage, &quot;https://app.tohure.assets&quot;)</pre><blockquote>¿Recuerdan lo que hablamos sobre loadDataWithBaseURL()? el método postWebMessage me acepta tanto el mensaje con puertos a enviar, como también el <strong><em>target origin</em></strong>, que me asegura el origen del mensaje a donde va a ser enviado, en este caso “https://app.tohure.assets”. Si lo hubiere trabajado con loadURL el origin hubiese sido <strong>null </strong>por lo que como parámetro de mi postWebMessage hubiese tenido que enviar algo como “*”, dicho de otra forma, acepta todo. Para una demo no hay problema, sin embargo en ambientes productivos siempre debemos tener cuidado con eso.</blockquote><ul><li>Ya habiendo configurado el puerto 1 (para escuchar mensajes entrantes) y enviado el puerto 2 (el que se envía a la web para la comunicación). Podemos enviarle mensajes a la web a través del puerto 1 por medio de <a href="https://developer.android.com/reference/android/webkit/WebMessagePort#postMessage(android.webkit.WebMessage)">postMessage</a> con un objeto <a href="https://developer.android.com/reference/android/webkit/WebMessage">WebMessage</a>.</li></ul><pre>port1?.postMessage(WebMessage(&quot;Saludos desde mi aplicación Android&quot;))</pre><h4>La Web</h4><ul><li>La web recibe el puerto enviado por Android a través del evento <strong>message</strong>. (No confundir con el <strong><em>onMessage</em></strong> de los puertos). Este evento fue disparado por Android cuando se ejecutó postWebMessage().</li></ul><pre>let appPort;<br><br>window.addEventListener(&quot;message&quot;, function(event){<br>  if(event.ports[0]) {<br>      appPort = event.ports[0];<br>      appPort.onmessage = onMessage;<br>  }<br>});</pre><p>Agregamos al objeto <strong>window </strong>de JavaScript el evento <strong>message </strong>con ayuda de <strong>addEventListener. </strong>Los puertos vienen dentro del evento (<strong>event.ports</strong>) por lo que podemos asignarlo a una variable y así trabajarlo de manera más eficiente.</p><ul><li>Una vez capturado el puerto, le asignamos un callback de onMessage, para <strong>escuchar</strong> cada vez que le lleguen mensajes a este puerto. (Tal como hicimos en Android con el Puerto 1 y setWebMessageCallback()); los cuales estarán almacenados dentro del campo <strong>data</strong></li></ul><pre>appPort.onmessage = onMessage;<br>        <br>function onMessage(event) {<br>  const messageApp = event.data;<br>}</pre><ul><li>Para poder <strong>enviar mensajes</strong> a la aplicación, debemos usar el mismo puerto con el que escuchamos pero ahora con el método <strong>postMessage()</strong></li></ul><pre>appPort.postMessage(&quot;Message from Web site&quot;);</pre><p>Y listo!</p><figure><img alt="Meme de ardilla diciendo que Al fin todo acabó" src="https://cdn-images-1.medium.com/max/516/1*8fqaUUJjPkLTjpmiiIHDPQ.png" /></figure><h3>Consideraciones finales (y muy importantes)</h3><blockquote>Si bien envié un mensaje super simple a la web en este ejemplo, en producción lo ideal es usar contratos o un token (de preferencia el contrato), valiendosé de Sealed Class en Kotlin y luego pasarlo a un texto JSON. Mas o menos así:</blockquote><pre>val contract = Contract(<br>  topic = &quot;payments&quot;,<br>  version = &quot;v2&quot;<br>  token = &quot;my-ultra-secret-token&quot;<br>)<br>val jsonString = Json.encodeToString(contract)<br><br>port1?.postMessage(WebMessage(jsonString))</pre><p>Y en web validar esos campos antes de si quiera asignar los puertos.</p><blockquote><strong>Importantísimo:</strong> La validación del origin se basa en un archivo local para este caso, pero en producción deben considerar puntos como HTTPS, dominio verificado, etc. “V<em>alidar el </em><em>targetOrigin es lo que realmente blinda tu app.&quot;</em></blockquote><blockquote>Cuando tu aplicación se cierre o tu Webview tenga que cerrarse para pasar a otra pantalla nativa, etc; no olviden cerrar los puertos para prevenir <strong>memory leaks.</strong></blockquote><pre>port1?.close()<br>port2?.close()</pre><p>Y esto porque WebMessagePort (objeto con el que creamos los puertos en Android/Kotlin), no es un simple objeto que vive y muere dentro de la JVM, recuerden que son <strong><em>Transferable Objects</em></strong>, recursos más complejos de bajo nivel que existen en la capa nativa (C++). El garbage collector limpiará el recurso en la JVM sí, pero el GC no tiene conocimiento de los recursos nativos que estén conectados a mis objetos de Kotlin, por lo que la “tubería” que creamos, puede que nunca se libere.</p><figure><img alt="Un meme de Doofenshmirtz diciendo: ¿y ya?" src="https://cdn-images-1.medium.com/max/324/0*svacJZHOaegLL1ik.jpg" /></figure><p>Sí, ¡Ahora si yap!.</p><h3>Demo</h3><p>Les dejo la demo de todo el ejercicio en Android.</p><p><a href="https://github.com/tohure/WebviewMessageChannel">GitHub - tohure/WebviewMessageChannel: Implementación de Message Channel Api en un webview Android</a></p><figure><img alt="Ejemplo de aplicación Android usando puertos." src="https://cdn-images-1.medium.com/max/480/1*DClHrdxRnJKc9JdHzBAoUQ.gif" /></figure><h3>Documentación</h3><p>Algunos links que me fueron muy útiles durante el desarrollo (sí, documentación, lean por favor, no todo es GPT por favor implementa esto aunque no sepa de que estoy hablando).</p><ul><li><a href="https://developer.mozilla.org/en-US/docs/Web/API/Channel_Messaging_API">Channel Messaging API - Web APIs | MDN</a></li><li><a href="https://developer.android.com/reference/android/webkit/WebMessagePort">WebMessagePort | API reference | Android Developers</a></li><li><a href="https://developer.android.com/reference/android/webkit/WebView#createWebMessageChannel()">WebView | API reference | Android Developers</a></li><li><a href="https://developer.android.com/privacy-and-security/risks/insecure-webview-native-bridges#risk-messagechannel-risks">WebView - Native bridges | Security | Android Developers</a></li></ul><p>¿Has tenido problemas de seguridad con WebViews en el pasado?. Cualquier duda, consulta o mejora al código por favor siéntanse libre de escribirme a mis redes.</p><p><a href="https://bio.link/tohure">Carlo Huaman</a></p><p>Gracias y hasta pronto.</p><p>Dios los bendiga.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c2bacdb6e17" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Actions Sunset | ¿Qué sigue ahora?]]></title>
            <link>https://medium.com/@tohure/actions-sunset-qu%C3%A9-sigue-ahora-e1faac8df4f4?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1faac8df4f4</guid>
            <category><![CDATA[actions-on-google]]></category>
            <category><![CDATA[google-assistant]]></category>
            <category><![CDATA[android]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Tue, 30 Aug 2022 18:19:42 GMT</pubDate>
            <atom:updated>2022-08-30T18:19:42.145Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Atardecer con el logo de Google Assistant como Sol." src="https://cdn-images-1.medium.com/max/1024/1*Q3I9dCoyusCrSbYwX4jFqg.png" /><figcaption>Atardecer en Salas, Lambayeque — Perú</figcaption></figure><p>Todo empezó un Google I/O 2016 cuando <strong><em>Sundar Pichai</em></strong> anunció <strong><em>Google Assistant</em></strong> al mundo junto a <strong><em>Mario Queiroz, </em></strong>hablando sobre el futuro de las <strong><em>IA </em></strong>y de cómo <em>Google Assistant</em> se integraría en nuestras vidas y nos ayudaría a realizar tareas del día a día (Y de hecho así fue 😊).</p><figure><img alt="Imágenes sobre la presentación de Google Assistant en el I/O 16" src="https://cdn-images-1.medium.com/max/1024/1*FHkoAQ2Fi61s1xJP8bXJtg.png" /><figcaption>I/O 16, el año que empezó todo</figcaption></figure><p>Durante el I/O 2017, <strong><em>Scott Huffman</em></strong> y <strong><em>Valerie Nygaardy</em></strong> anunciaron la integración con terceros de Google Assistant. Esto abría las puertas a que el asistente de Google no sólo tuviera integración con las aplicaciones propias de su ecosistema (Youtube, Maps, Gmail, Calendar, etc), si no también con soluciones propias de los desarrolladores y/o APIs de terceros. Sólo por poner un ejemplo, podías preguntarle algo al asistente de Google y enriquecer tu respuesta con información que tuvieras en tus bases de datos o de alguna API como de Divisas, Películas, Datos deportivos o incluso el API de Star Wars 🤣.</p><figure><img alt="Imagen referencial sobre lanzamiento de Actions en el I/O 17" src="https://cdn-images-1.medium.com/max/1024/1*fyCt0g3A0G_WU4Q_GUWLJg.png" /><figcaption>I/O 17, un gran año para los Google Assistant Developers</figcaption></figure><p>Desde entonces el desarrollo de <em>aplicaciones para Google Assistant</em> (aka <strong><em>Actions</em></strong>) se extendió de tal manera que podías encontrar Actions prácticamente para todo. Sumado a que plataformas como <strong><em>Dialogflow</em></strong> y<strong><em> Actions Builder </em></strong>ayudaban y aceleraban el desarrollo de Actions<strong><em>,</em></strong> logrando que todas las personas con mucho o poco conocimiento de programación pudieran crear sus propias Actions, ya sea con líneas de código, archivos CSV o sólo con un par de clicks en las plataformas mencionadas.</p><p>Pero…</p><p><a href="https://developers.googleblog.com/2022/06/Helping-Developers-Create-Meaningful-Voice-Interactions-with-Android.html">Helping Developers Create Meaningful Voice Interactions with Android</a></p><p>El 13 de Junio del 2022 (recordar esta fecha), Google lanzó un comunicado acerca de la suspensión de las <strong><em>Conversational Actions </em></strong>(aka Actions Sunset). Donde en resumen mencionan que ya no se dará soporte al desarrollo ni mantenimiento de nuevas o existentes Actions de terceros en la plataforma de Google.</p><p>La verdad sí dolió mucho leer esta noticia, pero todo siempre pasa por algo y en el caso de gran G, siempre hay un plan detrás de todo.</p><p>El artículo menciona algo muy interesante y es cómo los usuarios cada vez interactúan más y más con sus <strong><em>dispositivos Android </em></strong>a través de Google Assistant<strong><em>. </em></strong>Esto abre la puerta a nuevas posibilidades como la interacción de los usuarios con las aplicaciones en su teléfono a través de comandos de voz, simplificar tareas cotidianas con nuestro smartphone, gestión de información de forma rápida y sencilla, etc.</p><p>Esto no significa que las personas no interactúen con sus altavoces inteligentes, si no por el contrario, que cada vez buscan más y más información enriquecida a través de pasos más sencillos. Y si bien los altavoces inteligentes nos ofrecen información acerca de algo en específico, una aplicación en <strong><em>Android</em></strong> nos da una suite completa de toda una búsqueda, tareas, respuestas, transacciones, comunicación con otras aplicaciones que podamos necesitar. Y ¿qué mejor manera de acelerar toda este conjunto de características que con comandos de voz y Google Assistant?.</p><figure><img alt="Aplicación Adidas Running siendo invocada por comando de Google Assistant" src="https://cdn-images-1.medium.com/max/1024/1*M5bn7tp0IJSnOm4YVbI7Fg.png" /><figcaption>Addidas Running App Action — Fuente: Google Assistant Site</figcaption></figure><h4>“App Actions — Your Android apps, voice amplified”</h4><p>Dándole voz a tus aplicaciones <strong><em>Android</em></strong></p><p>Las <strong><em>App Actions</em></strong> no son un concepto nuevo, de hecho fueron presentadas en el I/O 2018. A diferencia de las <strong><em>Conversational Actions </em></strong>que se enfocan en el desarrollo de aplicaciones para Altavoces y pantallas inteligentes, las <strong><em>App Actions</em></strong> se enfocan en la interacción de las aplicaciones Android con el usuario a través de comandos de voz con Google Assistant.</p><p><a href="https://developers.google.com/assistant/app">App Actions | Google Developers</a></p><p>Esto le daba un nuevo enfoque a aplicaciones que ya tenían años en el mercado y que ahora podrían interactuar de forma más rápida y efectiva con los usuarios con simples comandos de voz.</p><h4>“Actions Sunset “— ¿Qué no podré hacer desde ahora?</h4><p>Volviendo a la realidad 😭…</p><p>Muchos se estuvieron o estarán preguntando: ¿Qué sigue ahora?, ¿Qué no podré hacer desde ahora?.</p><p>¿Recuerdan la fecha que dije no olvidar? (13 de Junio de 2022). Pues tenemos un año desde esa fecha, es decir hasta el <strong><em>13 de Junio de 2023,</em></strong> para:</p><ul><li>Descarga de información de nuestras <strong><em>Conversational Actions</em></strong> disponibles en la plataforma de <a href="https://console.actions.google.com/"><strong><em>Actions on Google</em></strong></a><strong><em> </em></strong>o<strong><em> </em></strong>a través del <a href="https://developers.google.com/assistant/conversational/quickstart"><strong><em>Actions SDK</em></strong></a><strong><em>.</em></strong></li><li>Ver información en la herramienta de <a href="https://developers.google.com/assistant/console/analytics">Actions Analytics</a>, sin embargo esta será dada de baja antes, el<strong><em> 13 de Setiembre</em></strong>.</li><li>Crear nuevas <strong><em>Actions </em></strong>o editar existentes<strong><em>, </em></strong>aunque algunas herramientas para su desarrollo comenzarán a estar obsoletas</li><li>Los usuarios podrán seguir usando Actions que estén desplegadas a producción hasta la fecha mencionada. Luego de ello ya no podrán acceder a ellas.</li></ul><p>Pueden encontrar más información en el siguiente link:</p><p><a href="https://developers.google.com/assistant/ca-sunset">Conversational Actions sunset overview | CA sunset | Google Developers</a></p><h4>¿Aún podré usar Google Assistant en otras plataformas?</h4><p>¡Claro!</p><figure><img alt="Meme con Bardock como protagonista sobre el Actions Sunset" src="https://cdn-images-1.medium.com/max/548/1*376hCIGHtC9es3qqyMU06A.png" /><figcaption>¡No todo esta perdido!</figcaption></figure><p>Si bien esto ha sido un duro golpe para las <strong><em>Conversational Actions</em></strong>, Google Assistant aún nos ofrece un amplio ecosistema donde podremos seguir desarrollando tales como:</p><ul><li><strong>App Actions →</strong> Las Actions para aplicaciones Android.</li><li><strong>Smart Home →</strong> Se enfocará en el desarrollo para los Google Home.</li><li><strong>Actions Web Content →</strong> Actions generadas a partir de markup en el contenido de la web.</li><li><strong>Media Actions →</strong> Feeds que permiten describir tu contenido multimedia en las búsquedas y el Asistente de Google, así como iniciar su reproducción.</li><li><strong>Assistant Android Features →</strong> Características para interactuar con elementos o compartirlos con ayuda del asistente de Google como <strong><em>Assistant Sharing</em></strong><em> o </em><strong><em>Read-it.</em></strong></li></ul><h4>Apreciaciones finales</h4><p>No busco decir que las App Actions reemplazarán a las Conversational Actions, sé que muchos (me incluyo) pasaban una tarde conversando con su <strong><em>Google Home</em></strong> (o Smart Speaker con Assistant) y ver cómo su código cobraba <strong><em>su propia voz </em></strong>gracias a Google Assistant<strong><em>, </em></strong>era algo hermoso 🥺😭.</p><p>Sin embargo hay que enfocarnos en lo que tenemos ahora y cómo esto podrá seguir evolucionando. Las Actions no nacieron si no hasta que primero Google mostrase su Google Home al mundo. Puede ser que tengamos que volver a esperar y ver cómo evolucionará este nuevo ecosistema <strong><em>potenciado por los teléfonos Android </em></strong>para dar una palabra final sobre el futuro de las aplicaciones conversacionales.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1faac8df4f4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gradle, Kotlin DSL, AppEngine & Actions on Google?  Why not? ]]></title>
            <link>https://medium.com/@tohure/gradle-kotlin-dsl-appengine-actions-on-google-why-not-90b3cefb6378?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/90b3cefb6378</guid>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[google-assistant]]></category>
            <category><![CDATA[google-app-engine]]></category>
            <category><![CDATA[actions-on-google]]></category>
            <category><![CDATA[gradle]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Tue, 17 Mar 2020 23:21:19 GMT</pubDate>
            <atom:updated>2020-03-23T06:31:53.229Z</atom:updated>
            <content:encoded><![CDATA[<h3>Gradle Kotlin DSL with AppEngine &amp; Actions on Google 🤓</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0QfPeJJYkb2n5qO60lvneQ.jpeg" /><figcaption>Elephants thanks to my 5-year-old daughter Emy :3</figcaption></figure><p>A little over a year ago, the Actions on Google library was released to the public in Java/Kotlin. And since then the creation of Actions has grown and improved a lot.</p><p>In this article we won’t talk about what an Action is or how great they are (Come on!!, who don’t like your assistant to do what you want 😆), but how can we create our service for our agents.</p><p>The project we’ll do with the help of Gradle, we could have done it with Maven, however the purpose of this post is to talk about the advantages of Kotlin vs Groovy and thus take advantage of the potential of Kotlin as a language.</p><p>Although we have documentation on how to implement and run our project in Java, there are a couple of things that need to be modified if we want it to work in Kotlin.</p><h3>Requirements</h3><ul><li><a href="https://www.oracle.com/technetwork/java/javase/downloads/index.html">Java JDK</a> (version 8 or higher)</li><li><a href="https://gradle.org/install/">Gradle</a> (version 5.1.1 or higher)</li><li><a href="https://cloud.google.com/sdk/docs/quickstarts">Google Cloud SDK</a> (gcloud CLI)</li><li>IDE (IntelliJ, VS Code, Sublime, etc) — <em>You could use any IDE you want, in this tutorial we will use IntelliJ for the screenshots (2019.3).</em></li></ul><h3>Creating Project</h3><p>We have two ways of working here, by the command line or by the IDE.</p><h4>By Command Line</h4><p>With a couple of commands we’ll have the project ready. Obviously we must have <strong><em>Gradle</em></strong> installed in our OS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*kIwIRbuPETpWECTG8TFzxw.png" /></figure><ul><li><em>cd</em> FOLDER_PROJECT (if you don’t have it you can quickly create it with the <strong><em>mkdir</em></strong> command and then enter it with the <strong><em>cd</em></strong>)</li><li>gradle init</li><li>select kind project (application)</li><li>select language (Kotlin)</li><li>select script DSL (Kotlin not Groovy)</li><li>set project and package name</li><li>enjoy!</li></ul><h4>By IDE</h4><p>Well, this way is a little longer (a couple of clicks), but still valid, in fact if you did not install Gradle manually, the IDE itself has a version of it for project development.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*-nTofFCriESHjEhzA2PMZw.png" /></figure><p>First, we start with our IDE and select to create a new project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*lrCcR3MpW5Y-3tMScD4V2A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*YEJ9aDxA3_0IX6V2uW49xQ.png" /></figure><p>Then, in the next screen, we choose the type of project that will be <strong><em>Gradle (1)</em></strong>, and in the options window we mark that we will use <strong><em>Kotlin DSL build script(2)</em></strong>, don’t forget this step that is one of the most important. And finally that the library to work with will be <strong><em>Kotlin/JVM (3)</em></strong>, if you have marked by default Java please uncheck it.</p><p>Then we just give our project a name and finish it. Enjoy!</p><p>Whether we worked it through the command line, or through the IDE, we end up with structures like these:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*gU6gHK9YGfwCbjzKiGF_tA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*4WKTV6vZIwZZqtWlZ-YtoQ.png" /><figcaption>Left: By IDE | Right: By Command Line</figcaption></figure><p>In the case of having done it by IDE, it generates a Java folder, but you can delete it, anyway we wont use it 😆. Don’t worry about the structure of your Gradle file, after all, we’re going to modify it anyway. 😇</p><h3>Kotlin DSL</h3><p>Let’s take a break. So, We just create the project in Kotlin and that’s all?. What is DSL and why is it important to mention it? A Wikipedia definition says:</p><blockquote><em>A </em><strong><em>domain</em></strong><em>-</em><strong><em>specific language</em></strong><em> (DSL) is a computer </em><strong><em>language</em></strong><em> specialized to a </em><strong><em>particular</em></strong><em> application </em><strong><em>domain</em></strong><em>. This is in contrast to a general-purpose </em><strong><em>language</em></strong><em> (GPL), which is broadly applicable across </em><strong><em>domains</em></strong><em>.</em></blockquote><p>In other words, DSL as a tool allows us to harness the power of a specific language so that we can execute the functions we want them to perform under commands or sequences defined by us.</p><p>When we start factoring code, we create functions that take care of specific tasks, simplifying lines of code, we are writing DSL.</p><p>And to see it even more clearly, we have these examples:</p><pre>val map = HashMap&lt;*,*&gt;()</pre><pre>map.<em>mapKeys </em><strong>{ //some code }</strong></pre><pre>map.forEach <strong>{ //some code }</strong></pre><pre>map.<em>filter  </em><strong>{ //some code }</strong></pre><p>Functions as <strong><em>mapKeys, forEach, filter</em></strong> are example of Kotlin DSL, because you could write loop structures with conditionals with different values, etc or just write for example <strong><em>filter function</em></strong>.</p><p>In fact, every time you refactor code, atomizing your functions, such as using <strong><em>extensions functions </em></strong><em>(more info </em><a href="https://kotlinlang.org/docs/reference/extensions.html#extension-functions"><em>here</em></a><em>)</em>, you are also creating your DSL code.</p><p>I’ll try to write a more detailed post about this later so I can go into detail about Kotlin-DSL.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/266/1*E2a5iedgref7uyO_WKkYvQ.jpeg" /><figcaption>I promise! Trust me!</figcaption></figure><p>But understanding a bit about DSL, we can say that if we want our code completely in Kotlin, we should change our classic <em>Gradle File</em> in <a href="http://wikipedia.org/wiki/Apache_Groovy">Groovy</a> for Kotlin.</p><p>When we created our project (whether by console or by the IDE), we were created with two files with the extension .kt: <em>build.gradle.kts </em>and <em>settings.gradle.kts.</em></p><p>Let’s work with <em>build.gradle.kts </em>first. Let’s remove everything that is there and replace it with the file that I leave you with below and the result would be something like this:</p><h4>build.gradle.kts</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/89edfa4832a9cf82a5bc17342b7bd2b7/href">https://medium.com/media/89edfa4832a9cf82a5bc17342b7bd2b7/href</a></iframe><p>Let’s take it one step at a time:</p><p><strong><em>The “plugins” tag </em></strong>allow us to add different plugins (duh! 😆) to our project as:</p><ul><li>Kotlin (in this case version 1.3.70)</li><li>War (In a classic Java project we package <em>jar</em>, or an <em>Android Archive[aar]</em>, but we’ll be working on <em>Web Application Archive [war]</em>).</li><li>AppEngine (We’ll talk about this plugin later)</li></ul><p><strong><em>The “repositories” tag </em></strong>indicates where the <em>plugins</em> will be obtained or where their content should be resolved. In this case we work <em>here</em> with <strong><em>Maven </em></strong>to resolve the <em>Kotlin</em> and <em>War</em> plugin, but what about AppEngine? I’ll explain in a minute. 😇</p><p><strong><em>The “dependencies” tag </em></strong>contains the libraries to be used in our classes such as:</p><ul><li>Kotlin</li><li>Actions on Google</li><li>Json Parser</li><li>Slf4j to logger</li><li>Servlet</li></ul><p>And last but not least:</p><p><strong><em>The “AppEngine” tag</em></strong> will allow us to define the parameters for the deployment of our application in AppEngine such as:</p><ul><li>Project ID</li><li>Version (to specify a custom version)</li><li>stopPreviousVersion (If we need to stop the previous version )</li><li>promote (For traffic to be automatically routed to the new version)</li></ul><p>However the use of this tag, is thanks to the plugin in the line 4 of our build.gradle.kts.</p><pre>id(&quot;com.google.cloud.tools.appengine&quot;) <em>version </em>&quot;2.2.0&quot;</pre><p>And here we have to pause again and talk about how Gradle handles its DSL plugin for Kotlin.</p><p>As you can see on this page <a href="https://plugins.gradle.org/plugin/org.gradle.kotlin.kotlin-dsl">here</a>. Gradle tells us how to correctly use our <strong><em>DSL plugin</em></strong> or otherwise the legacy version that we all know with <strong><em>BuildScript</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*OqT6tOKssPstBcA3sDfYvg.png" /></figure><p>This is the main reason why, unlike some Java projects that use <strong><em>buildscript</em></strong>, we go straight to the <strong><em>plugin</em></strong> tag. But as you can see, a basic condition to be able to use this DSL, is that the ID must be the same as the coordinates of the plugin. And since even I didn’t understand what I wrote 😆, I better show an example based on the image above:</p><pre>BuildScript Dependency</pre><pre>COORDINATE          :   NAME      :   VERSION<br>org.gradle.kotlin   :   plugins   :   1.3.4</pre><pre><br>Plugin</pre><pre>ID                  :   COMPONENT<br>org.gradle.kotlin   :   kotlin-dsl</pre><p>So as you can see, both the coordinate of our BuildScript Dependency and the ID of the plugin to use are the same (org.gradle.kotlin). But in the case of <strong><em>AppEngine</em></strong> that does not happen, both are different (ID and Coordinate).</p><pre>BuildScript Dependency</pre><pre>COORDINATE               :   NAME                      :   VERSION<br>com.google.cloud.tools   :   appengine-gradle-plugin   :   1.+</pre><pre>Plugin</pre><pre>ID                               :   COMPONENT<br>com.google.cloud.tools.appengine :   AppEnginePlugin</pre><p>Do you see the difference?</p><pre>com.google.cloud.tools<br> <strong><em>vs<br></em></strong>com.google.cloud.tools.appengine</pre><p>If you try to put the plugin directly, you’ll get an error that it couldn’t solve the dependency correctly. Something like this:</p><pre>Build file &#39;~/build.gradle.kts&#39; line: 1</pre><pre>Plugin [id: &#39;com.google.cloud.tools.appengine&#39;, version: &#39;2.2.0&#39;] was not found in any of the following sources:</pre><pre>- Gradle Core Plugins (plugin is not in &#39;org.gradle&#39; namespace)<br>- Plugin Repositories (could not resolve plugin artifact &#39;com.google.cloud.tools.appengine:com.google.cloud.tools.appengine.gradle.plugin:2.2.0&#39;)<br>  Searched in the following repositories:<br>    Gradle Central Plugin Repository</pre><p>So we could either do the entire BuildScript implementation in this file or we can tell it how to solve it correctly 🤓. And for this we will go to our second file today <strong><em>settings.gradle.kts</em></strong>.</p><h4>settings.gradle.kts</h4><p>In this file with the help of the <strong><em>pluginManagement</em></strong> specification, we’ll be able to indicate how to correctly solve the aforementioned plugin.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e88c887978b678121f4f5409650988c2/href">https://medium.com/media/e88c887978b678121f4f5409650988c2/href</a></iframe><p><strong><em>pluginManagement</em></strong> provides me a task called <strong><em>resolutionStrategy</em></strong> and it goes through each plugin that I’m working on at the moment. Just use a simple conditional of the plugin that I’m trying to invoke and then with the help of <strong><em>useModule</em></strong>, I indicate which is the correct module to use. I can receive very useful parameters like the <strong>ID</strong> and the <strong>VERSION</strong> that will allow me to know which is the plugin that I want to manipulate and the version that I want to bring. As there’s only one conditional for <em>AppEngine</em>, the other plugins like Kotlin and War will continue to be resolved by default.</p><h3>WEB-INF</h3><p>Since we have created a Kotlin project either by the IDE or by the command line, we’ve not generated our WEB-INF folder which will contain our appengine-web.xml file</p><blockquote>App Engine Java applications use a configuration file, named appengine-web.xml, to specify information about your app and to identify which files in the app&#39;s WAR file are static files (like images) and which are resource files used by the application. (Google Cloud Documentation)</blockquote><p>Since we don’t have the necessary folders to store this file, we must create them ourselves. The folders would go in this order: <em>webbapp → WEB-INF → appengine-web.xml</em>. Here is an example of how it should look.:</p><pre>MAIN_PROJECT_FOLDER<br>|<br>└── src<br>    └── main<br>        ├── kotlin<br>        ├── resources<br>        └── <strong>webapp</strong><br>            └── <strong>WEB-INF</strong><br>                └── <strong><em>appengine-web.xml</em></strong></pre><p>And within the file the necessary base configuration.</p><pre>&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&gt;<br>&lt;appengine-web-app xmlns=&quot;http://appengine.google.com/ns/1.0&quot;&gt;<br>    &lt;threadsafe&gt;true&lt;/threadsafe&gt;<br>    &lt;runtime&gt;java8&lt;/runtime&gt;<br>&lt;/appengine-web-app&gt;</pre><p>Everything worked so far only concerned the configuration of our project and the AppEngine plugins we needed. Now we can quietly develop our Fulfillment and Cloud Endpoint inside <strong><em>Kotlin</em></strong> folder.</p><h3>Fulfillment — Intent handler — DialogflowApp</h3><p>We want to make our agent do a thousand things. But deep down, what is this “fulfillment” that we want to see accomplished?</p><blockquote>When the Assistant invokes your agent, it acts as conversational fulfillment for your Action. The Assistant sends user utterances to your Dialogflow agent to match to an intent and respond back. Your agent matches the utterance to an intent and sends a response. The Assistant renders this response to the user, displaying it appropriately depending on the user’s device capabilities (audio and display output). (Dialogflow Documentation)</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/1*2V1Qq52MMCoQoP7jVUdpwQ.png" /><figcaption>Dialogflow Documentation: AoG Protocol</figcaption></figure><p>So for every instruction that we want our agent to respond to, there must be an <strong><em>Intent</em></strong> that is capable of responding within our <strong><em>Dialogflow agent</em></strong>. To do this we’ll create a class that extends from <strong><em>DialogflowApp()</em></strong>, and inside it, the intents that we want the agent to solve, <strong><em>Intents</em></strong> that should have been previously created inside the Dialogflow console.</p><p>As you can see I created a class called <em>MyActionApp</em> (you can call it whatever you want) that extends from <strong><em>DialogflowApp</em></strong>, inside this I have placed two intents (you can place as many as you want or need).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f84169fc5a760c1b1ab0ca468105f331/href">https://medium.com/media/f84169fc5a760c1b1ab0ca468105f331/href</a></iframe><p>The behavior of these <strong><em>Intents</em></strong> will be determined by respective functions for each. In order for our <em>Intents</em> to be resolved by these functions, they must be annotated with the <em>@ForIntent </em>annotation with the name as we have it in our dialogflow console.</p><p>Now what kind of answers can we add in these <strong><em>Intents</em></strong>? We’ll see that in a next episode.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/266/1*E2a5iedgref7uyO_WKkYvQ.jpeg" /><figcaption>I promise! I’ll make a post about it, in Kotlin, with gambling and everything else xD.</figcaption></figure><h3>Cloud Endpoint — HttpServlet()</h3><p>If we realize, our previous code will <strong><em>only be the processing engine of our Intents</em></strong>, but it does not have the ability to <strong><em>handle HTTP requests</em></strong>, so we need to create a controller capable of handling requests from our Webhook (HTTP/JSON webhook API).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ngw291x34SHPmsorQpIpgA.png" /><figcaption>Actions on Google invoking your fulfillment through Dialogflow (AoG Documentation)</figcaption></figure><p>Understanding the flow by which our Assistant will make the consultations with our agent, we have to provide a controller to handle these requests. To do this I created a class that extends from <strong><em>HttpServlet</em></strong>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7e514fec041e3fa281b45a8c99a74eda/href">https://medium.com/media/7e514fec041e3fa281b45a8c99a74eda/href</a></iframe><p>In this class, three things are worth noting:</p><ul><li><strong>MyActionApp <em>(line 4):</em></strong> It’s a reference to my <a href="#11f4">previously mentioned class</a> that extends from DialogflowApp, in order to be able to manipulate its answers later.</li><li><strong>doPost Function <em>(line 7):</em></strong> By the name of the method we can make a reference to what its function is. <strong><em>POST</em> <em>requests</em></strong> will pass through this method with the paylod of the <strong><em>Intent</em></strong> that needs to be resolved. After that, I pass this reference to my DialogflowApp class (line 12) in order to solve the queried Intent and give the appropriate response (line 14).</li><li><strong>doGet Function <em>(line 23): </em></strong>Although everything is handled by POST methods, we must not expose our <strong>GET</strong> method, so we can send an Unauthorized code (401), with a message that mentions it.</li></ul><h3>Deploying</h3><p>And finally we get to this part, we have all the ingredients and it’s time to deploy our Project in AppEngine.</p><p>Remember that one of the requirements at the beginning was to have <a href="https://cloud.google.com/sdk/gcloud"><strong><em>gcloud</em></strong></a> installed, right? Don’t forget that when installing it you must have initialized the SDK with your gmail account that has your project in Google Cloud (more information <a href="https://cloud.google.com/sdk/docs/quickstart-debian-ubuntu#initialize_the_sdk">here</a>).</p><p>From this point we have two simple ways to deploy our project, as at the beginning, either by console or by the IDE, and no matter how you create it, you can still do it the way you want.</p><h4><strong><em>By Command Line</em></strong></h4><p>Go to your project folder and run the gradle plugin with the <strong><em>appengineDeploy</em></strong> command</p><pre>gradle appengineDeploy</pre><h4><strong><em>By IDE</em></strong></h4><p>The IDE recognizes the available tasks of AppEngine for Gradle thanks to the plugin that we configured at the beginning. So the deploy task is in the <strong><em>gradle tasks panel</em></strong> of our IDE, under the name of <em>appengineDeploy</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3TEk2dR5FsJCBYH8WtJ9qg.png" /></figure><p>And we can prove that everything is going well through a simple POST or GET.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/1*psuX8KibyR7Ajgo_t04usg.png" /></figure><p>To test my service, I sent an empty POST, so it’s natural get an error when it’s a non-existent Intent that I am requesting. And in the case of GET we should have an unauthorized response as we configured in our <strong><em>doGet</em></strong> method above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*hnYlFthJobZWip1LoW2sww.png" /></figure><p>Here I leave you all the work done in a public GitHub repository.</p><p><a href="https://github.com/Tohure/KotlinActionSimple">Tohure/KotlinActionSimple</a></p><h3>Conclusions</h3><ul><li>Kotlin is a very powerful language, so we can take advantage of this language to quickly and robustly build our FulFillment.</li><li>Using Kotlin DSL allows us to harness the power of this language against Groovy, so the IDE engine is in charge of processing everything in Kotlin and not in Kotlin and Groovy.</li><li>Being a deployable project in AppEngine, you can use some other more powerful framework than just a simple HttpServlet, like <a href="https://spring.io/projects/spring-boot">SpringBoot</a>, which by the way is also available in <a href="https://kotlinlang.org/docs/tutorials/spring-boot-restful.html">Kotlin</a>.</li><li>The library is originally written in Kotlin so why not the rest of the code right? 🤓</li></ul><h3>References</h3><ul><li><a href="https://medium.com/google-developers/announcing-the-java-kotlin-client-library-for-actions-on-google-217828dead">Announcing the Java &amp; Kotlin client library for Actions on Google</a></li><li><a href="https://medium.com/@yoichiro/how-to-build-actions-with-java-client-library-on-google-appengine-2eef7a5bd402">How to build actions with Java Client Library on Google AppEngine</a></li><li><a href="https://stackoverflow.com/questions/48502220/how-to-configure-appengine-gradle-plugin-using-kotlin-dsl">How to configure AppEngine Gradle plugin using Kotlin DSL</a></li><li><a href="https://dialogflow.com/docs/integrations/actions/integration">Actions on Google integration | Dialogflow</a></li><li><a href="https://developers.google.com/assistant/conversational/overview">Conversational Actions | Google Developers</a></li><li><a href="https://docs.gradle.org/current/userguide/kotlin_dsl.html">Gradle Kotlin DSL Primer</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=90b3cefb6378" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dialogflow & Facebook Messenger, the story never told]]></title>
            <link>https://medium.com/@tohure/dialogflow-facebook-messenger-the-story-never-told-ac033202a667?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac033202a667</guid>
            <category><![CDATA[dialogflow]]></category>
            <category><![CDATA[facebook-messenger]]></category>
            <category><![CDATA[bots]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[messenger]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Fri, 16 Aug 2019 07:12:08 GMT</pubDate>
            <atom:updated>2020-01-12T05:26:21.429Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cyPV6kwY-p_-_PeosLhfiQ.jpeg" /></figure><p>A few weeks ago, I got a challenge to make a chatbot for Facebook messenger, so I didn’t think twice and said: Dialogflow Time!!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*EzVki5S0mPuV4LWjjyuyiw.gif" /><figcaption>When someone said me: Do it with Dialogflow!</figcaption></figure><p>Actually, on internet there are many tutorials on how to make chatbots for Facebook Messenger or even tutorials for Facebook and Dialogflow. So… why make another one?</p><p>One of the reasons is because I come from developing applications for Google Assistant (<a href="https://developers.google.com/actions/">Actions</a>) and <strong><em>use Dialogflow to create Actions</em></strong> <strong>it isn’t the same that create chatbots for some other platform like Facebook</strong>, there are a couple of things that change and we must take into account.</p><p>I won’t reinvent the wheel and I won’t do one more tutorial about how to integrate Facebook with Dialogflow, but I’ll give you a couple of tips that usually jump when using Dialogflow with Facebook, that I hope will be very useful to you when using both platforms.</p><p>Anyway, I leave here information on how to integrate Facebook Messenger with Dialogflow in case you require it:</p><p><a href="https://cloud.google.com/dialogflow/docs/integrations/facebook">Facebook Messenger | Dialogflow Documentation | Google Cloud</a></p><p>Well, let’s get to work…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*TAXL7sumsYsKYRQVPZUtqQ.gif" /><figcaption>Go ahead.</figcaption></figure><h3><strong><em>Tip #1 — </em>Dialogflow Fulfillment Library — </strong>Use it or not?</h3><p>I know this sounds strange, after all, the ideal would be to work with the library of this platform. However, it may be the case that you already have your own Webhook pro for Facebook running and you just want to add to it the Dialogflow functionalities or vice versa, you are starting an implementation for your chatbot and you want to delegate to Dialogflow the topics like authentication, reading, contexts, etc.</p><p>For example, if you started with a clean webhoook for your Facebook bot and in the road you added a dialogflow features, your webhook maybe looks like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/76b1ed0b8efa188c45e266358c240e49/href">https://medium.com/media/76b1ed0b8efa188c45e266358c240e49/href</a></iframe><p>And actually this approach is fine if you have already structured your project and you are already working directly with the documentation of Facebook webhooks. You can continue without any inconvenience.</p><p><a href="https://developers.facebook.com/docs/messenger-platform/getting-started/webhook-setup">Webhook Setup - Messenger Platform - Documentation - Facebook for Developers</a></p><h3>Tip #2 — Facebook Carousels with Dialogflow?</h3><p>The carousel is an ordered and sequential way of showing results in the same line of space. Achieving to give the user a pleasant experience. Although this control has been seen in several platforms such as Google Assistant, Facebook chatbots also have them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*78HQHH-yOjem7Au4z7_-rg.png" /><figcaption>source: Facebook Deevelopers</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*paLT9kXW9U1v0nDPUTDtaA.png" /><figcaption>Dialogflow Facebook Responses</figcaption></figure><p>If you see the Dialogflow platform, in Facebook Responses section, we only saw 5 types of answers and the carousel is not among them. Does it mean we can’t implement it from Dialogflow?.</p><p>No, not exactly.</p><p>Let’s take a little look at how many of today’s chat platforms work when building bots on them, under a Webhooks system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EKwtECFdD0EyqXV1" /><figcaption>source: Actions on Google Documentation</figcaption></figure><p>Answers, rich responses, permissions, data, etc, are sent under JSON schemes, allowing us to know the body of the requests of any element that we are working in our chatbot. When we add an element such as a card for example in Google Assistant, we are actually sending a body of this type:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*4tjRkcNGYiFts4SzRRvAHA.png" /><figcaption>source: Action on Google Documentation</figcaption></figure><p>Why am I telling you all this?, just in case you find an element that Dialogflow doesn’t handle natively, you can build it yourself through this option <em>(Custom Payload)</em>, either from the platform, or from code at the time of implementing your webhook.</p><p>So, going back to the carousels on facebook messenger, the documentation tells us that carousels are formed from something called “Generic Template”. The only thing we should do is to take that structure mentioned in its documentation and use it in Dialogflow’s Custom Payload option.</p><p>When you add a “Custom Payload” response, Dialogflow automatically gives you a preview of the body you have to build. And as I said before, we only have to use the structure that gives us the documentation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*mBWkV0G8HoE_48cQD3rcfg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*YygfuJ1LmilFTm4ptTnmMQ.png" /><figcaption>Dialogflow platform (left) — Facebook Developer Documentation (right)</figcaption></figure><p>At the end, we should have something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pqEA9Yw-cBcRKwwykyyH-A.png" /><figcaption>Dialogflow platform — Facebook Custom Payload</figcaption></figure><p>Only <strong><em>DON’T FORGET, </em></strong>Facebook’s carousel only supports a maximum of <strong><em>10 elements</em></strong>, otherwise, if you attach even 11 elements, your carousel will never appear.</p><p>At code level it’s similar, only using the Payload object from Dialogflow library. More information <a href="https://dialogflow.com/docs/reference/fulfillment-library/rich-responses#payload_%E2%87%90_richresponse">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/1*uXPUrEHp8vWVuZaScS4Ufw.png" /></figure><h3>Tip #3 — Facebook events in Dialogflow</h3><p>We know that Dialogflow works under <strong><em>Intents</em></strong>, which are activated by something that the user spoke or wrote in your chatbot. But what happens when we want to call an <strong><em>Intent</em></strong> under a user event that is not necessarily written or spoken?. It is in this section that we talk about <strong><em>Dialogflow Events</em></strong>.</p><blockquote>Events allow you to invoke intents based on something that has happened instead of what a user communicates. Dialogflow supports events from several platforms (like Google Assistant, Slack, and more) based on actions users take on those platforms. | <em>Dialogflow Documentation</em></blockquote><p>Well, let’s talk about the 3 most popular events for messenger bots.</p><h4><strong><em>FACEBOOK_WELCOME</em></strong></h4><p>If we look at the Dialogflow platform, in the events section, there is only one event for Facebook. Which is the <strong><em>FACEBOOK_WELCOME</em></strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*0duJZGapW1EGW6CLCOj7Eg.png" /><figcaption>Dialogflow Platform — Events Section</figcaption></figure><p>The <strong><em>FACEBOOK_WELCOME</em></strong> event can be triggered by the detection of words trained in the welcome intent, as well as by a trigger when starting a new conversation with a messenger bot for the first time with a button like “<em>Get Started</em>”. This one can be edited to trigger any specific different event you need. More information <a href="https://developers.facebook.com/docs/messenger-platform/discovery/welcome-screen/">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*kOV0JQckYSNmsW5n_dI73Q.jpeg" /><figcaption>Facebook Messenger | New Conversation (left) | Old Conversation (right)</figcaption></figure><p>In the case of the image on the left, whenever you start a new conversation, <strong><em>FACEBOOK_WELCOME</em></strong> will be launched, however in the image on the right, if you use some random word that is not in your training model of your INTENT WELCOME, the <strong><em>FACEBOOK_WELCOME</em></strong> event will never be triggered.</p><h4><strong><em>FACEBOOK_MEDIA</em></strong></h4><p>Do you know that through Facebook Messenger you can attach files to your conversation, such as images, audios, documents? (the latter only from the web version). And if you wanted to detect when a user uploads a file of these types to your conversation and get it? This is where <strong><em>FACEBOOK_MEDIA</em></strong> comes in.</p><p>The event as such does not appear eligible on the Dialogflow platform, but you can still write it and it will take it as valid.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/453/1*FnjAM8X-aRRISQoro1jJEA.png" /><figcaption>Dialogflow Platfrom — Events</figcaption></figure><p>This event will allow your <strong><em>Intent</em></strong> to trigger automatically when the user attaches some kind of file to your conversation. To get the file sent we have to go to the body of the response obtained by Dialogflow (body / originalRequest) in this order:</p><ul><li>originalDetectIntentRequest → payload →data →message →attachments</li></ul><p>If we put the above in a variable should stay this way:</p><pre>const _attachments = originalDetectIntentRequest.payload.data.message.attachments;</pre><blockquote>Don’t forget to always validate that objects are not null.</blockquote><p>This variable “<strong><em>attachments</em></strong>”, is returned as an array, because you can upload multiple files at the same time, but each shares the same structure, a field “<em>type</em>” that returns what we have in a “<em>file</em>”, a field <em>payload.url</em> that returns the url where facebook will host our file and then download it to some server.</p><pre>_attachments[0].payload.url =&gt; file url</pre><pre>_attachments[0].type =&gt; document type</pre><h4>FACEBOOK_LOCATION</h4><blockquote>The <strong><em>FACEBOOK_LOCATION</em></strong> event allows you to get a location from Facebook Messenger. Create an intent to request the location using a custom payload. | <em>Dialogflow Documentation</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/665/1*EX-_7tIZBLeMFXcQlBN65g.png" /><figcaption>source: Facebook Developers</figcaption></figure><p>Facebook handles a quick response that allows us to request the user’s location. Dialogflow can capture the interaction of this action as if the user gave permission or not to get their location through the <strong><em>FACEBOOK_LOCATION</em></strong> event. We can also obtain the data as latitude and longitude if the user provided it successfully. As with FACEBOOK_MEDIA, the data is inside originalRequest, payload and in this case, postback.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*NKobFBcz_MmnLxmQg8erNQ.png" /><figcaption>source: Dialogflow documentation</figcaption></figure><h3>Tip #4 — m.me Links</h3><p>Did you know that you can launch your bot’s conversations with the micro data you need to customize the start of a conversation from a short link?. Yes, you can, and these links are the <strong><em>m.me links</em></strong>.</p><blockquote><strong><em>m.me</em></strong> is a shortened URL service operated by Facebook that redirects users to a person, page, or bot in Messenger. You can use them on your website, email newsletters, and more. | Facebook Developer Documentation</blockquote><p>Let’s take the following scenario: You want someone to interact with your bot, and you want to send them specific information about a particular product, or about a particular article. You could share the link of your website, tell him to interact with your bot and then look for what he needs (no, I don’t think so), or you could give him an m.me link.</p><p>Let’s imagine you have a facebook page with the name DialogflowFriends, your facebook link would be more or less like this:</p><pre>https://www.facebook.com/DialogflowFriends</pre><p>The link that would open the chat on this page would be:</p><pre>https://www.facebook.com/messages/t/DialogflowFriends</pre><p>And the m.me that would be the short version of the previous one would be like this:</p><pre>http://m.me/DialogflowFriends</pre><p>Which will redirect you to the previous one, so far there is nothing out of the ordinary, is a simple “<em>short link</em>”. The great thing here is that to m.me you can add data as parameters inside the same url. If I wanted to share the id for example of a product or an article, I could do something like that:</p><pre>http://m.me/DialogflowFriends?ref=&lt;REF_PARAM&gt;</pre><p>And where can I capture this information?. For this we have two scenarios, when the user interacts for the first time with his bot or when he already has a conversation started.</p><h4>First-Time Use Scenario</h4><blockquote>If you have configured a “<strong><em>Get Started</em></strong>” button for your page, we will deliver the passed ref param as part of the postback event that is normally sent when the button is tapped. | <a href="https://developers.facebook.com/docs/messenger-platform/discovery/m-me-links/#reading_parameter"><em>Facebook Developer Documentation</em></a></blockquote><p>As Facebook says in the previous paragraph, when we launch a new conversation from a m.me link with parameters, this information at first instance, seems to disappear from the url, but actually this in the event that will be sent through the button “<strong><em>Get Started</em></strong>”, then this information you can capture in your code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/348/1*2jS77BpSiQ_DZ_Q7aDAZFA.png" /></figure><pre>const linkData = originalDetectIntentRequest.payload.data.postback.referral;</pre><p>As you can see, all the information is the “<em>referral</em>” object, where “<em>source</em>” is of type SHORTLINK, since it comes from a <em>m.me</em> link and “<em>ref</em>” contains the information that we send as parameter in the link.</p><h4>Existing User Scenario</h4><p>When you’ve already had a previous conversation with a Facebook bot, and you access it through an <strong><em>m.me</em></strong> link, the information comes through a Facebook event called <strong><em>messaging_referrals</em></strong>, so your webhook should be prepared to listen to it; but…and Dialogflow?</p><p>Unfortunately Dialogflow does not yet support this type of event, but it is not the end of the world. Well, it doesn’t support it natively, because if you have built your webhook cleanly as we talked about in the first <a href="#343d">Tip #1</a>, your webhook should be able to hear that event without problems, and then through the Dialogflow API you could explicitly invoke it.</p><p><a href="https://cloud.google.com/dialogflow/docs/reference/rest/v2/projects.agent.sessions/detectIntent">Method: projects.agent.sessions.detectIntent | Dialogflow | Google Cloud</a></p><p>With respect to the information, it arrives in the same “<em>referral”</em> object that we mentioned in the previous section.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*RIDFsS92oHgxZjB8aVkhcg.png" /><figcaption>source: Facebook Developers Documentation</figcaption></figure><h3>Tip #5 — Be careful with Payloads</h3><p>In <a href="#d1b4"><strong><em>Tip #2</em></strong></a><strong><em>,</em></strong> I told you about the advantage of using the “<strong><em>Custom Payload</em></strong>” response to build custom responses in case the Dialogflow platform doesn’t offer it natively. However, if you are using Dialogflow’s NodeJS library there are a couple of things you need to know, for example</p><p>You can’t send multiple Payloads because currently the library doesn’t support it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*3a3PM7cLOVvdfE_oYJJNAA.png" /><figcaption>No, you can’t</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/657/1*SUP8qlRRKcb-dGCmlxTlJQ.png" /><figcaption>Not this way either</figcaption></figure><p>In fact, this feature is still under discussion in Dialogflow’s Github threads.</p><p><a href="https://github.com/dialogflow/dialogflow-fulfillment-nodejs/issues/177">Multiple agent.add fails with integrations · Issue #177 · dialogflow/dialogflow-fulfillment-nodejs</a></p><p>But as I said before, it’s the end of the world? Well, no, not yet xD. Depending on the requirement you have to do, there are still some tricks you can apply.</p><h4>Idea N° 1</h4><p>Something similar to the previous tip, if you don’t use the library and you are building from 0 your webhook yourself, you could edit the response to force two payloads at the same time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/1*eEa2iVuVzOQMo4urwOWsGQ.png" /><figcaption>This way you could send two payloads at the same time.</figcaption></figure><h4>Idea N° 2</h4><p>But what if you already started using the library and would like to write a text and at the same time a carrousel?. Well, let’s remember that communication through Facebook is through webhooks, so it means that we ourselves could invoke Facebook service to send a message.</p><p><a href="https://developers.facebook.com/docs/messenger-platform/send-messages">Messaging - Messenger Platform - Documentation - Facebook for Developers</a></p><p>The previous page is summarized in sending the wished object through POST.</p><pre>https://graph.facebook.com/v4.0/me/messages?access_token=&lt;PAGE_ACCESS_TOKEN&gt;</pre><p>We already have <strong>PAGE_ACCESS_TOKEN</strong>, because we use it when configuring Dialogflow with Facebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*-Cvr-Jd3b6yQDQOZF_q2YQ.png" /><figcaption>Remember?</figcaption></figure><p>Then we need to indicate to whom we will send the message (recipient user), and for this we need to know the <strong><em>PSID</em></strong><em> (page-scoped ID)</em>.</p><p>But where do we get it? Well, Dialogflow gives us this value from the moment we start a conversation, we can find it within the parameters of the Dialogflow <strong><em>Generic Context </em></strong>under the name <strong><em>facebook_sender_id</em></strong>, or in the field of the response <strong><em>originalDetectIntentRequest</em></strong> as <strong><em>sender</em></strong> and its “<strong><em>id”</em></strong> value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*u0mDu-K2RF6q0WyO9eKGgA.png" /></figure><p>So we have the token, we have the <em>PSID</em>, what’s next? Send the message.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*yaInQe6je0iyQ_j3c3hI_Q.png" /><figcaption>source: Facebook Developers</figcaption></figure><p>Well this is easily encapsulated in a function, and now when you want to send a text and a custom payload together, you could do something like that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/1*oAxb4AqwHhkqm1sLkOYnkA.png" /><figcaption>It doesn’t necessarily have to end like this, you could make it better.</figcaption></figure><h3>Tip #6 — Facebook will not store information for you, Look for an alternative to storage</h3><p>If you come from using Dialogflow with Google Assistant, you will know that you can save information between turns of a conversation or save information on the device and be available in multiple conversations.</p><p>However the case of Facebook is different, there is no way to save on the Facebook platform information about the conversation. Even Dialogflow stores the information about the sessions for a maximum period of 20 minutes, after that, the information that you have stored in the contexts and parameters, will disappear.</p><blockquote>By default, output contexts expire after either five requests or <strong>20 minutes </strong>after its corresponding intent is matched. If the same output context is included in another intent, the context resets the counter and clock to five requests and <strong>20 minutes</strong>. | <a href="https://cloud.google.com/dialogflow/docs/contexts-input-output#lifespan">Dialogflow Documentation</a></blockquote><p>I don’t have a code extract for this tip, because here, you could apply different types of solutions depending on the requirements you have. For example, personally, I use <strong><em>Firestore</em></strong> to create data structures according to the chat session I’m in or according to the user who started the chat. And as <strong><em>Firestore</em></strong> is a realtime database, it is very useful when bringing and writing information into it while the user writes to the bot at the same time.</p><p><a href="https://github.com/dialogflow/fulfillment-firestore-nodejs">dialogflow/fulfillment-firestore-nodejs</a></p><blockquote>I may be forgetting a couple of things, or there may be other cases that have happened to you and not to me. If I find any other I promise to add it to this post. =)</blockquote><h3>References</h3><ul><li><a href="https://cloud.google.com/dialogflow/docs/contexts-input-output#lifespan">Input and output contexts | Dialogflow Documentation | Google Cloud</a></li><li><a href="https://developers.facebook.com/docs/messenger-platform">Messenger Platform - Documentation - Facebook for Developers</a></li><li><a href="https://dialogflow.com/docs/reference/fulfillment-library/rich-responses">Dialogflow</a></li><li><a href="https://firebase.google.com/docs/firestore">Firestore | Firebase</a></li><li><a href="https://miningbusinessdata.com/four-ways-to-set-parameter-values-in-dialogflow/">Four ways to set parameter values in Dialogflow - Mining Business Data</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac033202a667" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Debugging our Cloud Function in Real Time]]></title>
            <link>https://medium.com/@tohure/debugging-our-cloud-function-in-real-time-adfbf4ce3385?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/adfbf4ce3385</guid>
            <category><![CDATA[google]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[cloud-functions]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[actions-on-google]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Sun, 27 Jan 2019 04:12:47 GMT</pubDate>
            <atom:updated>2019-01-27T04:12:47.926Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rvIU5t6tu4XWS-o_75u0tQ.jpeg" /></figure><p>When I’m think about “debugging some of my Android applications”, the first thing that comes to mind are the breakpoints that I can use thanks to its IDE during the execution of my code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ngqpQsiiAhNKAMBndcC6w.png" /></figure><p>Well, it happens that I also develop applications for Google Assistant, what if in that case deployed my webhook to a Cloud Function? How would debugging in this case? I can think of many ways, but one of them is to use the potential of the Google Cloud where it is hosted and thus be able to use more of its great tools.</p><p>I’m talking about <strong><em>Stackdriver Debug</em></strong>, this powerful tool will allow us to have breakpoints of our code in real time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/763/1*0ryt1Gwn3IoMpmTnDiXkfA.png" /></figure><p><strong><em>Stackdriver Debug</em></strong></p><blockquote>Feature of the GCP that lets you inspect the state of a running application, in real time, without stopping or slowing it down. Your users are not impacted while you capture the call stack and variables at any location in your source code. You can use it to understand the behavior of your code in production, as well as analyze its state to find those hard-to-find bugs (<a href="https://cloud.google.com/debugger/">Google Stackdriver Debugger</a>)</blockquote><p>As I said at the beginning, I can debug some of my Android applications with the help of Android Studio, however this slows down many times my application, so Stackdriver Debug is a great option for our applications, more when we can test them in real time without slow them down.</p><p>I know many probably would say: Hey, it comes by default when we make a project with App Engine. I know, but in this case, we will not deploy our webhook in App Engine, we will use a Cloud Function, but… GCF still doesn’t support <strong><em>Stackdriver Debug</em></strong> (not officially yet) so, how can we work it?</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FRrVzUOXldFe8M%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fexcited-yes-nicolas-cage-RrVzUOXldFe8M&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FRrVzUOXldFe8M%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="326" frameborder="0" scrolling="no"><a href="https://medium.com/media/97bb073e80f0a68cee57c6202dbddc11/href">https://medium.com/media/97bb073e80f0a68cee57c6202dbddc11/href</a></iframe><p>We’ll divide this little tuto into two parts: Code and Console (GCP). Let’s go first with de Code:</p><h4>Step 1 — Code</h4><p>In our code, locate us in the folder of our <em>function </em>and install <strong><em>debug-agent</em></strong>:</p><pre>npm install @google-cloud/debug-agent</pre><p>Make sure the package was installed correctly by checking our package.json.</p><pre>&quot;dependencies&quot;: {</pre><pre><strong>&quot;@google-cloud/debug-agent&quot;: &quot;3.0.1&quot;,</strong></pre><pre>&quot;actions-on-google&quot;: &quot;2.5.0&quot;,</pre><pre>.<br>      .<br>      .</pre><pre>}</pre><h4>Step 2 — Code</h4><p>Now we need to create a Stackdriver Debug instance in our code, which we can handle as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*prJQ096vfQy48imgAdXxhw.png" /></figure><p>The method .start() initialize debugger and now is ready to work!!!</p><p>It is <strong>important </strong>that this instance is <strong>at the beginning of our function</strong>. Imagine that we have our Action in an index.js file, then it would look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MwHFmF7Qa8lnsXsuX6cOXg.png" /></figure><h4>Important to know!!!</h4><p>The Debug Agent sync the snapshot information almost always <strong><em>instantaneously</em></strong> but weird exceptions that might delay up to 40 seconds.</p><p>So, <em>Debugger</em> for give enough time, the tool provides us with a function called <strong><em>isReady()</em></strong>, which returns a promise when it’s ready.</p><p>The documentation mentions that it has to be implemented before exiting the cloud function, in order to give the debugger enough time to synchronize the snapshots so, don’t try to implement before any call of conv.ask or conv.tell since you could have a WebHook error for timeout or malformed response.</p><pre>debug.isReady().then(() =&gt; {</pre><pre>    debugInitialized = true<br>    console.log(&quot;Debugger is initialize&quot;)</pre><pre>});</pre><h4>Step 3 — Code</h4><p>Finally, only need to upload our code to any repository like GitHub, GitLab, Bitbucket, etc and obviously, deploying these recent changes to our Cloud Function</p><pre>git push origin *somewhere*</pre><pre>firebase deploy</pre><p>Now it’s time to work over Google Cloud Platform!</p><h4>Step 1 — Console</h4><p>First, go to the Google Cloud console and select the project that is associated with your deployed Cloud Function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ODe9DoXRmcu1hduwI66gqA.png" /><figcaption>Google Cloud Platform</figcaption></figure><h4>Step 2 — Console</h4><p>Open the left side panel and scroll down to the debugger option and click.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/1*zCZKAgWGUF-5iZ6meaz5gw.png" /><figcaption>Google Cloud Platform — Stackdriver Debugger</figcaption></figure><p>Once we are in the Stackdriver Debugger panel, see that all options look empty or invalid, and this happens because the project still doesn’t detect the Stackdriver Debugger module in our GCF, but no problem, they will appear in a few steps later, hehe.</p><p>First, let’s identify what we have in this window.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y-34quqewvxXnma0I5HwfA.png" /><figcaption>Google Cloud Platform — Stackdriver Debugger</figcaption></figure><ul><li><strong><em>1 — </em></strong>It allows us to select any of the origins that we have defined previously, or in other words, the base source code that we will use to define the breakpoints to analyze in our project.</li><li><strong><em>2 — </em></strong>If we have more than one application deployed to our project, we can select the one we want to debugging. At this point, we observe that we don’t have any application no matter how many clicks we make here. This is because Stackdriver is not yet natively supported in our GCF.</li><li><strong><em>3 — </em></strong>This could be the best part that Stackdriver Debugger offers us. We can choose from where we will update our code source, it gives us different integration options like GitHub, BitBucket or if that’s not enough, we can upload the local files of our computer if we don’t want to give any access to account.</li><li><strong><em>4 — </em></strong>In this panel we will be able to see all the breakpoints that we have added to our code and have a follow up of each one.</li><li><strong><em>5 — </em></strong>By breakpoint detected in the real time of our code, the stored information of the methods as well as variables or identified objects, will appear in this panel and thus see the value of each one.</li></ul><h4>Step 3 — Console</h4><p>In this step, it’ll be necessary to add the source code to track of our Cloud Function. Although as I mentioned before, we can add it from any repository or we can upload it from our local machine.</p><p><em>For the purposes of this example, I’m using a code which represents an exercise from</em><strong><em> Action on Google’s CodeLabs</em></strong>. If you haven’t seen them yet, here’s a little summary about them.</p><p><a href="https://medium.com/orbismobile/my-experience-with-google-assistant-codelabs-and-why-you-should-also-take-them-right-now-8e7a77719d2c">My experience with Google Assistant Codelabs and why you should also take them right now!</a></p><p>Well, go to “Select source” from GitHub and click it!.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mnb0ULmr6cl31xaEWIkrXg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4p_eu2vptD6UdKKD8YQd7w.png" /><figcaption>Select the source to work</figcaption></figure><p>Remember that if your first time giving the source, Google Cloud ask you for permissions for your repository with a little modal like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/1*7jxdKObpIhT5gdmZDhWKig.png" /></figure><p>The “<strong>*</strong>” it’s depend repository that you choiced (GitHub, GitLab, BitBucket, etc). Don’t worry and just Authorize this part. 🤗</p><p>Selected the source, the panel should look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jJ5pS38gsAyB1AhUqyp8lg.png" /><figcaption>GCP — Stackdriver Debug</figcaption></figure><p>After you have selected the origin, the tree of our project will appear in the left panel. So now we select the file where is the code we want to debug, in our case index.js.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7qY-7YbGKUBUKxjKvV07EA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/879/1*eHX2jES6n5FvYPILAK4gqw.png" /></figure><p>The code we are looking at can be found in the following repository.</p><p><a href="https://github.com/Tohure/AssistantAppTraining">Tohure/AssistantAppTraining</a></p><h4>Step 4 — Console</h4><p>Having selected the data source, it is time to select the application to debug.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i0Ysvis0pLaXJCcmJWRL1A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*ImVB8wjzqi9Euknz5RLEhw.png" /><figcaption>GCP — Stackdriver Debug</figcaption></figure><p>It may happen that even after you have done all of the steps above, your application may still not appear.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/586/1*py62ngn7ehnpxJ5gyu3Xmg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/622/1*VMbUVf0TFAXBR6g4sz03wg.png" /></figure><p>To avoid this, just invoke our Action once with the debugger library implemented, and the google console, just give it a refresh. 😉</p><h4>Step 5 — Console</h4><p>In this last step, we only need to put our breakpoints, where debugger will create a snapshot each time that point is invoked or used, with this we will have the information in real time when that fragment of code is called, the respective information will appear in the right panel.</p><p>To put a breakpoint we need to click on the number of the line of code we want to track:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*i5eYlBDOzAX-3y9Bs6LHrA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*gZnSF8iLeBXkiuah2JQePA.png" /></figure><h4>Step 6— Console</h4><p>That’s all!! Time to enjoy your Debugger in real time, notice that now the panels are filled with information relevant to your function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fI5pMUHFe1_HhyEisBYpbQ.png" /></figure><ul><li><strong><em>1 — </em></strong>The line of code invoked at that moment, is colored in order to recognize in which part of the code we are at the moment that our Action is running.</li><li><strong><em>2 — </em></strong>The right panel is filled with the respective information of the variables and functions called up to that moment and the value they have in that point of the program.</li><li><strong><em>3 — </em></strong>The snapshots history will show us the time each snapshot was invoked, as well as the information they had at a given point in time, since the same line of code may have called at two different times and each have different values.</li></ul><h3>Conclusions</h3><ul><li>Stackdriver captures exceptions in real time.</li><li>Stackdriver doesn’t slow down your application while it is being used.</li><li>Stackdriver can use any code resource, from GitHub to a project folder on your local machine.</li><li>Stackdriver is very easy and fast to implement.</li></ul><h3>Special Greetings</h3><p>I want to thank my friends <a href="https://medium.com/u/c23421e64eb5">Yoichiro Tanaka</a>, <a href="https://medium.com/u/e5a2c4437d5b">Amanda Cavallaro</a> and <a href="https://medium.com/u/41a6565d9e5b">Allen Firstenberg</a>, they supported me a lot to understand some concepts that I had very vague. Otherwise I wouldn’t have been able to create this little post. Thanks guys!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FRNUq6qOJwcsp2%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FRNUq6qOJwcsp2%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FRNUq6qOJwcsp2%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="443" frameborder="0" scrolling="no"><a href="https://medium.com/media/74d9fe4ee50ed4f113721ea4358f2ef9/href">https://medium.com/media/74d9fe4ee50ed4f113721ea4358f2ef9/href</a></iframe><h3>References</h3><ul><li><a href="https://github.com/googleapis/cloud-debug-nodejs#using-stackdriver-debugger-on-google-cloud-functions">googleapis/cloud-debug-nodejs</a></li><li><a href="https://medium.com/google-cloud/stackdriver-debugger-on-google-cloud-functions-d49a6ee5f12c">Stackdriver Debugger on Google Cloud Functions</a></li><li><a href="https://actions-on-google.github.io/actions-on-google-nodejs/index.html">actions-on-google</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=adfbf4ce3385" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Picking up specific information to your conversation | Entities in Dialogflow]]></title>
            <link>https://medium.com/orbismobile/picking-up-specific-information-to-your-conversation-entities-in-dialogflow-3353170f52a6?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/3353170f52a6</guid>
            <category><![CDATA[actions-on-google]]></category>
            <category><![CDATA[dialogflow]]></category>
            <category><![CDATA[google-assistant]]></category>
            <category><![CDATA[google-actions]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Sun, 02 Sep 2018 05:32:15 GMT</pubDate>
            <atom:updated>2019-02-20T17:26:59.947Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/878/1*qmHF72KomHL24ABAMx9How.jpeg" /></figure><p>Imagine yourself talking to someone and at the end of the conversation she/he says: Sorry, I didn’t pay attention, I know you were talking to me but I forgot what you were talking about… What happens then? Conversation doesn’t flow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*tZCV7bV69CVNAkdLflSwXQ.gif" /><figcaption>See how important it’s to pay attention?</figcaption></figure><p>It’s for this reason that it’s not only important to know that a conversation has started, it’s also important to collect important information from it, in order to generate alternative responses or flows when talking with our users.</p><p>And how can we collect information ?. Through <strong>Entities</strong> in Dialogflow. But what is an <em>Entity</em>?</p><p>They’re powerful tools used to extract required information from a natural conversation. Any important data you wish to obtain from the conversation must have its respective entity created. It’s not necessary to create entities for every possible word or phrase, only for the necessary data.</p><p>All right, enough talking and let’s get started!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*iIsswMIJPkf-5g_hKDWRnA.gif" /></figure><h4>How do I use Entities?</h4><p>When we create an Intent, Dialogflow gives us the “Training phrases” section where we will put the entries that will activate the Intent for the user.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*aIN7i0hg5CA4e8cxzz3eWw.png" /><figcaption>Dialogflow Console — Intent</figcaption></figure><p>How do we use an Entity here? Well, let’s imagine we make an application for our bike shop (Yeah, I love bikes 😆) and one of the entrances is: My road bike is broken, can I go tomorrow?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*OzAHyhmYyHumJQ6t8UBRNw.png" /><figcaption>Example entrance — Dialogflow Console</figcaption></figure><p>Even this part is more than enough to activate the Intent, but we don’t collect data in any way. There are <em>two ways</em> to collect them (Example Mode &amp; Template Mode).</p><h4>Example Mode</h4><p>The <em>example mode</em> is activated by default in your “Training phrases”. In this mode, words that match your entities will be highlighted in color, and if they don’t match the first time, you can always double-click on the word or phrase you want to be recognized and choose the corresponding entity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*oR1e-me3lU79ACDLVuv4xA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*t8ysVta5QN3f1OIJBAwr8Q.png" /></figure><p>So far so good, but what if I need to validate more than one date? It would be something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*Isj84xNYyvjYnl4g-cUUcg.png" /><figcaption>No, I don’t think so, and there are still dates to go.</figcaption></figure><p>So what can we do, one option is to finish putting all the available dates on the calendar and then finish when we are old or . . .</p><h4>Template Mode</h4><p>To access this mode, we need to click on the icon on the left and see it change to a “@”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/1*07QjDFWri_44q2FbCO-R8Q.png" /><figcaption>Click on that icon.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/1*dDvp8I5HyMNhG1UkMbPtAg.png" /><figcaption>Perfect!</figcaption></figure><p>Then, instead of writing a lot of sentences about different dates, we can do something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/765/1*BhpAAlbmKUCDSreX6niPAw.png" /></figure><p>As you can see, now instead of writing the matches, type the name of the entity (invoked by @) and then Dialogflow will match all the words and phrases of that entity.</p><p>Whichever mode you use to match the words, in both cases you need to define the <em>parameters</em> where you will store the data obtained by the entity.</p><h4>Parameters</h4><p>Through a parameter, we can connect a word or phrase to a specific entity. We can place them in two sections, <strong><em>“Training Phrases”</em></strong> section and <strong><em>“Action &amp; Parameters”</em></strong> section.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*H72Ww6bglAtZXEOYkllOlw.png" /><figcaption><strong><em>“Training Phrases”</em></strong> section</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*jjfTLNeeL2B7tqlUhpP2DA.png" /><figcaption><strong><em>“Action &amp; Parameters”</em></strong> section</figcaption></figure><p>In both sections we observe that there are 3 equal fields: <em>Parameter Name, Entity and Value</em>.</p><ul><li><strong><em>Parameter Name: </em></strong>It’s the name given as a variable to take the matched value of the entity. It is very important to define them at the beginning, because if a value is not paired with a variable, the console will give them an error when trying to display a simulation. Normally the console automatically puts a name, but if you want you can change it, remember that the name is defined after the colon of the entity. You can change it from any section: <strong><em>“Training Phrases”</em></strong> section or <strong><em>“Action &amp; Parameters”</em></strong> section.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/785/1*hoJbwlWJtWprHGcr9NWpVw.png" /></figure><p>However, always remember to homogenize the names, having different names from the same entity could cause troubles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*5PPdrH7ivpml9RFpWhpTlg.png" /><figcaption>Please avoid this, believe me, you’ll thank me.</figcaption></figure><ul><li><strong><em>Entity</em></strong>: Name of the entity that matched. Dialogflow has three types of entities:</li></ul><pre><strong><em>System Entities</em></strong><br>Pre-built entities provided by Dialogflow that facilitate the most commonly used concepts in a conversation.</pre><pre><strong><em>Developer Entities</em></strong><br>If you need to recognize more specific or less common words or phrases in your conversation, you can create your own entity to match the data.</pre><pre><strong><em>User Entities</em></strong><br>Entities that can be defined for a specific user session. User entities and sessions last 30 minutes.</pre><ul><li><strong><em>Value: </em></strong>The <em>Value,</em> is the value obtained from the coincidence between a word or phrase and an entity. Note that in this section you must be very careful of the options that value can offer us, these are <strong><em>default</em></strong> and <strong><em>original</em></strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*6JhjeXe0JiR5_0fmmo8GZA.png" /></figure><p>But, What is the difference between choosing a default and original value?. Let’s take the date as an example, if we ask something like: My road bike is broken, can I go tomorrow? Where <strong><em>“tomorrow”</em></strong> is matching for a Entity. If we invoke an Intent with that phrase, the result obtained would be something like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*8xEMSU1dyq0PVwH3Ca0hEQ.png" /></figure><p>But if we mark the original value, the result would be the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/213/1*eOUJ9HUANky9EHy13UySiw.png" /></figure><p>If we choose to choose the original option, the Intent will literally return what the user said and if we leave it in default, the Intent will return the value interpreted by the Entity. And what kind of values can we obtain according to the Entity? Well, that depends on what entity we’re working, for example:</p><h4><strong><em>System Entities</em></strong></h4><p>As you know, Dialogflow manages its own entities that greatly help to identify common data in a conversation. This means that we do not have to create any entities of any kind on our own in the Dialogflow console. Among some of them we have:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/462/1*QkMGWXUlfMY4GNDKqkG7kQ.png" /><figcaption>Dialogflow Entities by Dan Imrie-Situnayake</figcaption></figure><p>There are many other system variables, you can find more information about them in the following <a href="https://dialogflow.com/docs/reference/system-entities"><em>link</em></a>. Also there are many ways in which we can use the entities of the system, here are some examples:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*tlqUvyH-opmc-Wap3uoF_A.png" /><figcaption>Dialogflow Entities by Dan Imrie-Situnayake</figcaption></figure><p>During this post we have used @sys.date a lot, so let’s use another one like @sys.unit-length. If we receive “<em>ten meters</em>”, as a result we’ll obtain: {“amount”:10, “unit”: “m”}; unless we use the “original” property we saw earlier and as a result we would get “ten meters” again.</p><blockquote>Remember that all values are returned by Dialogflow as JSON</blockquote><h4>Developer Entities</h4><p>If we have everything in the <em>System Entities</em> that Dialogflow handles most commonly, what about what it doesn’t handle? There is no such thing as @sys.bikes. So. . . What should we do? Create our own entity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*1daLcdVnVAxOZ013oAW8Tg.jpeg" /></figure><p>First go to the left navigation bar and then go to the Entities section, then click on the Create Entity button and let’s get started!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PFl8M-qov3ByC4tKr9YCAw.png" /><figcaption>Dialogflow Console</figcaption></figure><p>The first thing we will find is a screen where the following main data are required: <strong><em>Entity Name, Primary Value (Reference Value) and Synonyms.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*d4sXtgJTenaxemN4o8_YoQ.png" /><figcaption>Entities Panel — Dialogflow Console</figcaption></figure><ul><li><strong><em>Entity Name: </em></strong>The name that will identify the entity we are going to create.</li><li><strong><em>Reference Value: </em></strong>Main word that will serve as a tag to group other phrases with the same semantic value.</li><li><strong><em>Synonyms: </em></strong>Similar words that represent the same semantic value as the <em>Reference Value</em>.</li></ul><p>So, let’s take the bicycle shop as an example, the first thing to create is the entity that contains the types of bikes, let’s call it the @bike-type.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*FdWLJOmmK84BNfgl4KStkw.png" /></figure><p>We now have an entity called <strong><em>bike-type</em></strong> with three values, Road, Mountain and Cross bike, each with their respective synonyms. And now with this, we can identify the variables we need when the user asks us something.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*gyW0ysUD8ApOQnOgliN9Ug.png" /></figure><p>Remember that these entities will never return the synonym, on the contrary the Reference Value, unless you select <em>value.original</em> as we saw <a href="#e100">above</a>. But wait!!, we’re missing a feature of our entities: <strong><em>Automated Expansion</em></strong><em>.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/1*ER2OMWFS99j1SyJIFVmsXA.png" /><figcaption>Automated Expansion Option — Dialogflow Console</figcaption></figure><blockquote><strong><em>Automated Expansion - </em></strong>Allows your agent to capture words beyond those defined in your entities list by analyzing both the position and meaning of entities in your intents example phrases. by Dan Imrie-Situnayake</blockquote><p>It’s a great feature that it practically allows our entity to grow on its own. Obviously it’s always better to have all the possible options mapped out, but in case we are missing one, Dialogflow can help us with that.</p><p>What about if the entity we want to create is a little more complex? A case that happened to me some time ago is that I wanted to capture the value of my country’s currency, the Peruvian Sol (Sol Peruano - PEN). Perhaps you could tell me: for coins there is <strong><em>@sys.unit-currency</em></strong>. So let me tell you that my currency is not yet supported by Dialogflow. I hope someday yes 😆. Then what could I do? I could use a system variable to always capture a numerical value near the word “sol” or “soles” (Spanish words that in English might sound like Sun). But I would leave aside the word “Sol” or “Soles”, and if someone said something that begins with “Sol” and didn’t refer to the currency of my country? Well, I found a way out of my dilemma. <strong><em>Composite Entities.</em></strong></p><h4><strong><em>Composite Entities</em></strong></h4><p>When you need to conceptualize an entity with more than one attribute, something more complex than a system entity or a normal entity created by us, you can build your <strong><em>Composite Entity.</em></strong></p><p>Let us take the example of the currency of my country: the “Sol Peruano” (PEN). There are only two possibilities to say it, in singular or plural (<em>Sol</em> or Soles). An important point to take into consideration is the amount, it would be too long to create an entry for each possible number so I can use a system entity to capture the amounts. With that in mind, we can build our <em>composite entity</em>.</p><p>First we created an entity that houses the variations of the “Sol” currency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*lSpKVV41U7x56Y-QOdID-g.png" /><figcaption>“Centimos” is a variation for lower values of the “Soles”, such as cents of dollar.</figcaption></figure><p>Then, we create an entity that contains the types of the “Soles” and the amounts that the request may contain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*mgrnmc624JV3kvIUNrJN-w.png" /></figure><p>In order to create your composite entity, you must <strong><em>turn off synonyms</em></strong>. After that you refer the entities you need, for the amount we will use a “number” system entity, and to identify the currency, we mention the entity we created previously. Let’s not forget to put a name for the variables of each entity, otherwise we won’t be able to get the values of what the user says correctly.</p><p>Summary, our entity uses:</p><ul><li><strong><em>Mount</em></strong>: Value of type @sys.number. This will represent the numerical amount of my currency.</li><li><strong><em>Currency: </em></strong>Value of type @pen-types. This will represent the type of currency I am using.</li></ul><p>If we invoke our intent with a phrase like: My road bike costs 20 soles? we can get the following values:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/276/1*yrtqbSpoq_1Ex6tVZ659mg.png" /></figure><p>Where currency parameter is “Sol” and the mount parameter is 20.</p><p>And last but not least we have the <strong><em>User Entities</em></strong>. These entities are updated per user session, so they usually only last 10 minutes. These types of entities need to be created from the console for then you can modify their content. However, you can also create them from the API. For example, let’s assume you want to list hospitals, restaurants, museums or something specific about a city. If we first chose London we would have a list of museums, for example, very different if we had chosen Paris or Peru. For more information about this type of entities visit this <a href="https://dialogflow.com/docs/reference/agent/userentities">Link</a>. I think I’ll need an exclusive post for this section. 🙄 🤔</p><h3>Conclusion</h3><p>It’s good that we have well defined our Intents for everything that the user can say, however it’s just as important to define our Entities, since as I said at the beginning, it’s useless to know that your other party is talking if you don’t understand what it says or do not understand the important information of what it wants to say.</p><h3>References</h3><ul><li><a href="https://dialogflow.com/docs/entities">Entities overview | Dialogflow</a></li><li><a href="https://googlecloud.tips/tips/022-dialogflow-composite-entities/">Composite entities with Dialogflow * Guillaume Laforge</a></li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FkzdL6GxJ_WY%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DkzdL6GxJ_WY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FkzdL6GxJ_WY%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/869562f69fd9ddc4cb5ed74e9c8f226d/href">https://medium.com/media/869562f69fd9ddc4cb5ed74e9c8f226d/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3353170f52a6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/orbismobile/picking-up-specific-information-to-your-conversation-entities-in-dialogflow-3353170f52a6">Picking up specific information to your conversation | Entities in Dialogflow</a> was originally published in <a href="https://medium.com/orbismobile">OrbisMobile</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Playing with Surface Capabilities in your Actions]]></title>
            <link>https://medium.com/orbismobile/playing-with-surface-capabilities-in-your-actions-8f1e5ade335b?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f1e5ade335b</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[actions-on-google]]></category>
            <category><![CDATA[google-assistant]]></category>
            <category><![CDATA[firebase]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Sun, 12 Aug 2018 22:38:54 GMT</pubDate>
            <atom:updated>2018-08-12T22:38:54.156Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j35EYT_k6PZKLmO7T2ex_A.png" /></figure><p>One of the most important points when doing our Actions is to clearly define your conversational flow. What paths to take if the device with Google Assistant has or doesn’t have a screen?, has a web browser or not?, has sound at least?. All this is important to map to define the flow correctly and give the best user experience. It’s at this point that we enter the concept of <strong><em>Surface Capabilities</em></strong>.</p><p>You can define the features your application will support. Whether the use of a screen, sound, or web browser will be restrictive. All this was defined from your Action on Google panel. You can access it from your project panel, clicking on <strong><em>Surface</em></strong> in the left panel, and then on the right screen choose what conditions your application will support.</p><h3><strong><em>Surface Capabilities</em></strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ufxDfmT5FHbfgnVSHpm8RQ.png" /><figcaption>Action on Google Platform</figcaption></figure><p>For example, in the image, I have put that none of the above features are necessary or restrictive, so my application will be available on any device with Google Assistant.</p><p>But for example, if I defined that a screen is necessary to display information, then my application would no longer be available for any version of Google Home. And don’t worry, the console will always be warning you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UwcIeLl65hy50hhpRE8Iqw.png" /><figcaption>Action on Google Platform</figcaption></figure><blockquote>All right, but how would you handle this on runtime? 🤔</blockquote><p>We can also manage the <strong><em>Surface Capabilities</em></strong> from our code/platform and treat them in runtime.</p><h3>Runtime Surface Capabilities</h3><p>When we work on our actions and we consider the runtime behavior, we found different kids of situations. For example, always display information on visually appealing cards, or using multimedia controls or even having the same information structure regardless of how it’s presented to us.</p><p>The <a href="https://developers.google.com/actions/assistant/surface-capabilities#runtime-surface-capabilities">documentation</a> tells us about the 3 most common cases that can occur when we work on our actions:</p><ul><li>Response branching</li><li>Conversation branching</li><li>Multi-surface conversations</li></ul><h4>1 - Response Branching</h4><p>Imagine you want to make an application to tell you the time, weather or even the current dollar value. If you have a Google Home and you ask him about the current value of the dollar, He can tell you audibly without problems. And if you do it from a phone, you could attach a picture to your answer that complements the answer.</p><p>In other words, in this kind of case, neither the flow nor the structure of the information must changes by the type of surface.</p><p>The Actions library provides us with constants that help us identify the type of surface we have available. We can access them with the help of the conversational interface “<strong><em>conv</em></strong>”, and the use of the “<strong>surface</strong>” class.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zW9ZD6EM3EeWx1uGypkFBg.png" /><figcaption><strong>surface.capabilities.has</strong> function | Actions Library</figcaption></figure><p>Let’s make a small example and define an attempt, where we ask the current value of the dollar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zfm3xLK5bhSD0GaqpvpEXg.png" /><figcaption>actions.capability.SCREEN_OUTPUT</figcaption></figure><p>As you can see on line five, I’m checking to see if my current device where the action was invoked, is supporting some kind of screen, through the <strong><em>conv.surface.capabilities.has</em></strong> function, where the parameter to perform this verification is: <strong><em>actions.capability.SCREEN_OUTPUT</em></strong></p><pre><strong>conv.surface.capabilities.has</strong>(<em>&#39;actions.capability.SCREEN_OUTPUT&#39;</em>);</pre><p>So now, the situation is simpler than it seemed at first, now we simply have a situation of truth or falsehood. If it’s true, we put the desired text plus any image <em>(line 8)</em> that we need to enrich the answer otherwise, if we don’t have a screen, we simply proceed to speak the answer. It’s important noting that in either case, the flow of the app as well as its response have not changed, the only thing that changed is how the information was shown.</p><h4>2 - Conversation Branching</h4><p>Well let’s face it, there are certain application flows that can be a bit long and tedious with only voice commands, while on a screen that receives all the user inputs, the process is shorter and faster.</p><p>Imagine for example that you need to give to the user a set of options to choose, each one with different descriptions, then you have two options, read them one by one next to their descriptions 😐, and in case the user doesn’t understand well, you should have to repeat all the options again 😱 😨 or, you can show everything in a carousel of cards with all the necessary information and navigate between them until you choose the desired option with peace of mind.</p><p>Then, it means that we have to divide certain flows of our application according to the type of <strong><em>Surface</em></strong> that the <strong><em>assistant</em></strong> has available at the moment.</p><blockquote>So, how do I indicate what “Intent” should react according to the type of Surface? 🤔</blockquote><p>With Contexts!!!. Just as our “<em>Intents</em>” may receive <em>contexts</em> defined by ourselves or some other particular “<em>Intents</em>”, the Assistant also launches its own <em>contexts</em> according to the type of device where it’s running, so that it can be easily recognized. We have the following types of <em>contexts</em> defined with respect to the <em>Surface</em> that can be launched:</p><ul><li>actions_capability_audio_output - The device has a speaker.</li><li>actions_capability_screen_output - The device has an output display screen.</li><li>actions_capability_media_response_audio - The device supports playback of media content.</li><li>actions_capability_web_browser - The device supports a web browser.</li></ul><p>We just have to choose under what <em>context</em> we want our <em>intent</em> to be trigger and place it as an <strong><em>input context</em></strong> in our Intent in the Dialogflow console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*hbXc_bprZWc-7AEaYwt_Jw.png" /><figcaption>Dialogflow Panel — Intents</figcaption></figure><p>For example, here a Intent that will only trigger on surfaces with screens using the <strong><em>actions_capability_screen_output</em></strong> context:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/1*KKkeGfhuP8F-DGpRdYjnvw.png" /><figcaption>Action on Google Documentation</figcaption></figure><h4>3 - <strong>Multi-surface Conversations</strong></h4><p>Now we’ll get into the case that we’ve started a conversation from Google Home where we’re looking for information about something in particular, the conversation has gone very well, but in the end, you need to see the resulting information on some screen. What are you doing? We move the conversation somewhere else. Literally 😆.</p><p>To move the conversation to another site, we first need to verify that it is available. In this case, we want to move from a situation where we don’t have a screen to show data, to a place where we do.</p><p>Let’s verify that you have a screen available to work, otherwise you will not be able to continue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LefTcOe676lXWit7LU-llg.png" /></figure><p>Using the function <strong><em>conv.available.surfaces.capabilities.has() </em></strong>we make sure we know if there is a screen available on the current user where to send the data we need, such as their Android or iOS phone. The only function that asks is what type of surface we are looking for, in our case SCREEN_OUTPUT.</p><p>If the above function returns true, it means that if we have a place to send our data but, how do we do it?, using the <strong><em>NewSurface()</em></strong> class.</p><blockquote><strong><em>NewSurface() — </em></strong>Requests the user to switch to another surface during the conversation. (<a href="https://actions-on-google.github.io/actions-on-google-nodejs/classes/conversation_question.newsurface.html">Action on Google Documentation</a>)</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gb6LbKsvLiA5XgfYctVBpA.png" /></figure><p>For create a <strong><em>NewSurface</em></strong> request, we need 3 parameters:</p><ul><li><strong><em>Capabilities:</em></strong> What kind of capabilities we look at for. In this case, only SCREEN_OUTPUT</li><li><strong><em>Context: </em></strong>The Assitant will ask permission (obviously) to send you data to your phone. <strong><em>Context</em></strong> gives a general idea of what will be sent when the assistant asks you if it has your permission to send the information to a new screen.</li><li><strong><em>Notification: </em></strong>This is the title of how the notification will appear on our phone.</li></ul><p>You can start trying this, either from the emulator or from your own Google Home, but it has to be an Assistant without a screen in order to test the features of move the conversation to another device.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zjZgzhCEN57EUpytnx6pkA.png" /><figcaption>Action on Google Platform Simulator</figcaption></figure><p>As you can see, I’m using the emulator but with the <strong><em>Surface</em></strong> setup in Google Home to simulate the startup of our app in a assistant without screen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/1*u9sS0LGxXWW0iY-AmFZLAw.png" /><figcaption>Surface Speaker (e.g. Google Home)</figcaption></figure><blockquote>Please note that both your Simulator or the Google Home that you’re using, and the device where you want to send the notification, must both be configured with the same Google Account in order to work together.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a46tnDuJorbThuKeNHGfgA.png" /><figcaption>Status Bar — Android Device</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5-3-RJxb75NNX3v5KD7QQA.png" /><figcaption>StatusBar expanded — Android Device</figcaption></figure><p>If you gave permission in the conversation to send the information to your phone, a notification should have arrived on your device, in my case I’m using my Android Device.</p><p>Remember that the title of your notification was defined earlier in your code.</p><p>And we launch our new screen requirement and that’s it all? Well, no. We need to catch this <strong><em>event</em></strong> in the Dialogflow console through an <strong><em>Intent</em></strong> with the actions_intent_NEW_SURFACE event, because when the user clicks on the notification, you must manipulate what should happen next.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*XdOz0JdSF7u3hJo2fJ7TxA.png" /><figcaption>actions.intent.NEW_SURFACE in Dialogflow Console</figcaption></figure><p>Then we can handle it through our webhook when it is invoked.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-FmYMPY_boyGpwTZeJxaYQ.png" /></figure><p>First I verify that the status of the new request is fine (line 2), and then I paint the result as I would like, for this example I used a <a href="https://developers.google.com/actions/assistant/responses#browsing_carousel">BrowseCarousel</a>. This time I am certain that I have a screen where I can show a more enriched answer.</p><p>And that’s all, now we have a conversation that started in a Google Home but continued on our phones.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*qPkvEGrgGqrR-1psh24kzQ.gif" /></figure><h3>Conclusions</h3><ul><li>Before developing your application, it’s always good to <strong><em>map the entire flow of your application</em></strong>. This will make it easier to understand in what situations you need a screen or just sound in your application.</li><li>Remember that you can limit <strong><em>Surface Capabilities</em></strong> from your console or at runtime with the help of the SDK in your webhook.</li><li>Always try not to interrupt the conversation to the user, give them alternate flows when you don’t have a <strong><em>Surface</em></strong> available, but always in the end, give the user what they came to look for in you.</li></ul><h3>References</h3><ul><li><a href="https://developers.google.com/actions/assistant/surface-capabilities">Surface Capabilities | Actions on Google | Google Developers</a></li><li><a href="https://actions-on-google.github.io/actions-on-google-nodejs/classes/conversation_question.newsurface.html">NewSurface | actions-on-google</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f1e5ade335b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/orbismobile/playing-with-surface-capabilities-in-your-actions-8f1e5ade335b">Playing with Surface Capabilities in your Actions</a> was originally published in <a href="https://medium.com/orbismobile">OrbisMobile</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Considerations when making External requests in your Actions]]></title>
            <link>https://medium.com/orbismobile/considerations-when-making-rest-requests-in-your-actions-a98e5dc35dd9?source=rss-763dfde193e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/a98e5dc35dd9</guid>
            <category><![CDATA[google-assistant]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[firebase]]></category>
            <category><![CDATA[google-actions]]></category>
            <category><![CDATA[actions-on-google]]></category>
            <dc:creator><![CDATA[Carlo Huamán]]></dc:creator>
            <pubDate>Fri, 10 Aug 2018 02:50:17 GMT</pubDate>
            <atom:updated>2018-09-26T05:03:38.218Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*81zL0JnD-yRevch7QbqPyQ.jpeg" /></figure><p>Probably during the development of your first Actions (App for Google Assistant), you added thousands of intents, different kinds of answers, language support, and everything you could, until you said, okay, it’s time to connect to some external API.</p><p>There are many ways that you can work it, as well as many libraries, many ways, many other good practices. I don’t pretend to be a definitive list, just an opinion on the things I’ve been through and would like to share. Well, let’s get started!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FxTkcELz3rjpLDDRAOI%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FxTkcELz3rjpLDDRAOI%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FxTkcELz3rjpLDDRAOI%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="181" frameborder="0" scrolling="no"><a href="https://medium.com/media/303471110c3284eee20ee60a86c77658/href">https://medium.com/media/303471110c3284eee20ee60a86c77658/href</a></iframe><h4><strong>1 — Separate your constant and common config</strong></h4><p>Regardless of whether you use a third-party library or native libraries, there will always be a base url where you can make your requests, it is good to have this information in a separate object.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*yJqSM3D4Tcv55o6TxYmPbA.png" /></figure><p>And then, you can use in any part of your code, something like this:</p><pre>url: ApiConstant.BASE_URL</pre><h4>2— Javascript is Synchronous, work with promises</h4><p>Nowadays exists many and different libraries for make request in NodeJs, so It’s important to know which one to use based on its features. And I consider one of the most important is that it works with Promises or make them yourself.</p><p>Why?, Because remember:</p><blockquote>JavaScript is a synchronous, blocking, single-threaded language. That just means that only one operation can be in progress at a time. (<a href="https://medium.com/@kvosswinkel/is-javascript-synchronous-or-asynchronous-what-the-hell-is-a-promise-7aa9dd8f3bfb">Kait Hoehne</a>)</blockquote><p>So, it’s probably that you may have problems when you run a request and it takes a while for it to respond, until then your Action will launch a <strong><em>Timeout Exception</em></strong>. But with promises you can solve it. Here are some interesting libraries.</p><p><strong><em>Work with Promises</em></strong></p><ul><li><a href="https://www.npmjs.com/package/axios">Axios</a> — Promise based HTTP client for the browser and node.js</li><li><a href="https://www.npmjs.com/package/got">Got</a> — Simplified HTTP requests</li></ul><p><strong><em>Work without Promises (remember here make your owm promises)</em></strong></p><ul><li><a href="https://www.npmjs.com/package/http">HTTP</a> — The standard library</li><li><a href="https://www.npmjs.com/package/request">request</a>- Simple to use</li></ul><p>I know there are many others available, remember that you are always free to choose the one that best suits your needs. 🙂</p><h4>3 — console.log() A great ally 😎</h4><p>I know that the Google console gives us a lot of help with its <strong><em>response</em></strong>, <strong><em>debug </em></strong>and <strong><em>error</em></strong> panel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*NkF9MUI5OIcmz2pgWhkPqg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pyx1_rVi3CYmqU0hV0bIGQ.png" /><figcaption>Response &amp; Debug Tabs in Action on Google Console</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aaEs9gxDdTP3bkNs2lY0DQ.png" /><figcaption>Error Tab in Action on Google Console</figcaption></figure><p>But sometimes we need more information in order to see in detail what really happens behind our code. All you need is a <strong><em>console.log(</em></strong>whatever you need<strong><em>) </em></strong>to be able to look more closely at details that may go unnoticed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/571/1*dqGjhYuUC2usprVJiLkxGQ.png" /><figcaption>Hi, I’m a console.log()</figcaption></figure><p>And where do I see the logs? Well, you can choose from the <strong><em>Google Cloud Platform</em></strong> or <strong><em>Firebase Console. </em></strong>Remember to use a <strong><em>tag</em></strong> to help you filter them quickly, I always use Tohure, it’s my nickname 😆.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1hADDPyQopmuSLfaNhlEOg.png" /><figcaption>Google Cloud Plantform</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_qYe_VDMT2mM7Z_ZzB9KxQ.png" /><figcaption>Firebase Console — I prefer this one, don’t tell anyone.😄</figcaption></figure><p>Actually at this point, you may use the console.log() not only when working with request, you can use it anytime actually, I just wanted to mention it in case I forgot.</p><h4>4— Dont repeat the same query, only save the first result</h4><p>I know it’s strange that I say that, but I’ve seen some people repeat the same query just so that the Assistant can give the result again in case the user needs it again. However this represents another query, another expense and another request, remember time is money.</p><p>So what can I do? Save the data in the session. How? With the <strong><em>conversationToken.</em></strong></p><blockquote>The <a href="https://developers.google.com/actions/reference/rest/Shared.Types/AppResponse#FIELDS.conversation_token">conversationToken</a> field is a string that contains an opaque token that is recirculated to the Actions every conversation turn. (<a href="https://developers.google.com/actions/assistant/save-data">Actions on Google Documentation</a>)</blockquote><p>So all we need to do is keep our <strong><em>small data </em></strong>on our object. And I say <strong><em>small</em></strong> because it’s for variables, small objects, some small list or other, it isn’t a database that can store Terabytes of information. There are other solutions for this, but I will talk about them in another post.</p><p>You can access <strong><em>conversationToken</em></strong> through the conversational interface “<strong><em>conv.data</em></strong>” and then type the names of the parameters that you want to store.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/571/1*N6sn_nfHN9ri9BzIhTjoGw.png" /><figcaption>conversationToken -&gt; conv.data</figcaption></figure><h4>5 — And last but not least. Your Plan</h4><p>Your request doesn’t work even though you’ve already checked everything? It’s likely that the error is on the other hand.</p><p>If your webhook is built on your own server or service, and depending on how it is configured, then no problem. But if you’re using <strong><em>Firebase Cloud Functions </em></strong>to make your webhook, it’s best to update your plan from Spark to Blaze.</p><p>Why? Spark plan does not allow requests to external APIs, only between Google products. It’s in the fine print.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/573/1*eHfhbPaHApzOHuKDQTHLHQ.png" /><figcaption>That’s right, it’s there.</figcaption></figure><p>But don’t worry, if you’re new to the console, Google will give you a free year to use it as you please. Also, the charge for each request is not much unless your application becomes popular worldwide. Well, in that case, if we had to think of another solution 😅.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/1*_UTyPBCjR0IfELEu6N_W4Q.png" /><figcaption>Always remember to use Blaze</figcaption></figure><h3>Conclusion</h3><p>And that’s it, I’m not a Javascript expert, but several of the situations mentioned in this post, they’ve happened to me and my friends. So I wanted to tell them in case anyone needs them. I’m missing some, so please write me your comments to improve this little ToDo List.</p><p>See you next time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a98e5dc35dd9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/orbismobile/considerations-when-making-rest-requests-in-your-actions-a98e5dc35dd9">Considerations when making External requests in your Actions</a> was originally published in <a href="https://medium.com/orbismobile">OrbisMobile</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>