Introducing IdeaSource: Open Source platform for open innovation

By | IdeaSource

We are pleased to announce that we’ve released our latest pet project, called IdeaSource. It’s a very basic MVP (stressing the minimum part) of a cross-platform app to enable crowdsourcing of ideas, that we used internally as a sandbox to research React Native development front to back.

IdeaSource home-page.

Why IdeaSource?

Coming up with good ideas can be hard, especially after a while when echo chambers tend to appear in teams, groupthink takes over and creative solutions are left out of the conversation. We wished to introduce an easy way to share and manage ideas generated by a larger pool of people than those directly tasked with coming up with a solution, providing mechanisms for everyone to vote to sort the proposed ideasIt’s about opening the discussion an enabling everyone to participate in a creative process, both in the generation of ideas, and in their sifting/sorting by means of likes.

How does it work?

Ideas are gathered around challenges, which can be problems, aspects or topics a group or organization may need to fix or improve.  A challenge can be anything ranging from the next feature/product to invest in, to what could be the new perk in the office, to creative ways to address pollution in a municipality.

In order to create a challenge you must enter a description, upload an image and set a deadline. Challenges can be either private or public. Private challenges are only available to users registered using the same corporate email domain, while open challenges are open for every user in IdeaSource. We may review this in the future, but for v1 this is the way it works.

Once a challenge is submitted, everyone can start contributing ideas, and votes for ideas already shared.

Main features

  • Challenge creation and management: Create open or closed challenges scoped only to users that share your email domain. Screenshot of the creation of challenges in IdeaSource
  • Idea sharing: Propose ideas for the challenge you feel closer to.Screenshot of the creation of ideas in IdeaSource
  • Reacting to ideas: Navigate and react to ideas with ease. Screenshot of the navigation of ideas in IdeaSource
  • Everything organized in your own profile: Check out the ideas you have created or liked, and the challenges you shared within your personal account.

Where can I get it?

The app can be downloaded from the App Store and Google Play Storeas well as it can be tried from the web browser. Regarding the code base, IdeaSource is meant to be an open source project for open innovation in organizations, so that it can be forked and modified, and for custom versions to deployed. In the following weeks we’ll be sharing the github code repo. In the meantime, we would love you to try it and provide feedback!!

ARKit 2: The Good, the Bad and the Ugly

By | Augmented Reality

Image of an iPhone showing a virtual pokemonARKit 2 is the latest framework from Apple that allows developers to work with Augmented Reality in iOS. When it launched in 2018, ARKit 2 brought new features and improvements in comparison with the previous version, but yet it has a long way to go. Our dev team spent some days researching and playing with it in order to find out how far it could go, and here are the conclusions.

Which things does this framework do well? And which ones not so much? Let’s take a look at the Good, the Bad and the Ugly of ARKit 2.

The Good about ARKit 2

Plane detection technology

ARKit 2 from stock includes many different tools in order to create a rich AR experience. One of them is the ability to detect horizontal and vertical planes by using the device’s camera and tracking what is known as features points. When the framework algorithm recognizes these points aligned in a real-world flat surface, it returns this information as an ARPlaneAnchor object, which contains data of that plane and represents the link between the real one and its virtual counterpart to be tracked throughout the AR session.

This information includes the general geometry of the detected plane, along with its center and dimensions, these being updated as new information from reality is discovered by the device. On supported iOS devices (with A12 or later GPU), ARKit attempts to classify detected planes into the categories of walls, floor, seats, ceiling, tables or none if no of the listed corresponds.  This additional data that the algorithm reports to the application might be of interest, for instance, to spawn certain objects on floors and not over every detected horizontal plane, allowing a more engaging AR experience.

Spawning virtual objects with their relative movement

As part of an enhanced augmented reality experience, virtual objects should behave as closely as possible to real ones. In particular, relative movement and parallax effects allow this greatly, simulating that objects are placed in the world itself and not just on the device’s screen with no relation to its surrounding. ARKit 2 is very successful at this task, tracking the user’s movements throughout the AR session and adjusting the virtual objects positions in consequence, relative to a set of initial coordinates.

Screen recording from an iPhone showing a spawned virtual Squirtle

This requires almost no effort at all by the developer’s hand, with the template being created as soon as you start a new project, showing an object (a spaceship asset) placed in the scene with the capability of letting the user move around it and see it from all sides. Therefore, this is a very strong feature of ARKit 2 especially considering how easy it is to work with.

Recognizing exact images

With our research of the ARKit 2 framework, we looked for solutions for image recognition in order to add virtual objects, like texts or even videos to identified images (see the example below). A use case for this feature could be showing information about a company when its logo is recognized. ARKit has a built-in image recognition system to which you can feed digital images that will be then recognized in the real world. SceneKit then allows you to perform certain actions once the image has been recognized by the software. This was accomplished with ease because, at the moment of loading the images, Xcode possesses an interface that analyzes them and gives the developer advice in regards of the image’s quality to ensure a more precise recognition. For instance, characteristics like size, colors or even design are taken to consideration.

After running several tests and following Xcode’s feedback, we could experience that the system does a reliable comparison of each image and makes sure they match exactly. Having said this, pictures such as landscapes with a wide variety of colors and shapes are identified in a good way as they are processed by their color histogram. Problems appear with simpler ones, something that we’ll discuss later in this post.

Realistic physical characteristics of an object

One of the approaches that we wished to cover was the ability of virtual objects to interact with each other and to have behaviors based on real-world physics. ARKit 2 brings the feature of adding physical characteristics to objects, like defining its mass. For instance, setting mass in virtual objects with oval bases such as a boat makes that when you move the plane where it is sustained the object will start to stagger.

Furthermore, the framework is capable of applying physical forces to virtual objects generating, in consequence, a reaction from it like moving forward or even falling if it is pushed away from the base-plane. This kind of physic properties improves the realistic appearance of virtual objects and the AR immersion.

Screen recording of an iPhone showing a virtual Squirtle falling from a table

Integral and adaptive architecture

In terms of its architecture, it might be worth mentioning that ARKit is similar (regarding its functionality into classes division) to other technologies that involve 3D object interaction, such as those pertaining to game development as the Unity engine. The concepts and terminology the framework handles, like Nodes, Anchors, PhysicsBodies, Shapes, among others, are present in some way and can be mapped to other technologies the developer might be more familiarized with, which is convenient in terms of speeding the learning process, pretty steep either way.

Apart from that, through our experimentation process, we found that it offers some possibilities of being integrated with other frameworks or libraries. In particular, for instance, a UIView (from iOS’s UIKit) can be placed in an augmented reality context, as a result of detecting a plane or image enabling then for buttons to be set in the user’s view and allowing interaction in this context or playing videos on some surface. Besides, when testing image recognition, it is possible to integrate ML Core which is an interesting option when extending the capabilities of ARKit in this sense is needed.

The Bad about ARKit 2

Plane recognition is not always accurate

 Even though the process of recognizing horizontal and vertical planes works fine in most cases, there are some edge cases where it doesn’t. When this happens, the feature points (that allow to identify planes) detected by the camera in the real world are not always “visible” because of certain circumstances of the context, such as the lighting, colors, and shadows.  For instance, a white wall or an uniform-color table don’t generate the specific characteristics needed for the camera to represent the feature points and to be able to work with. Take a look at the image below:

Images of a white table used as an example to ARKit plane recognition

We executed several tests where the goal was to identify a completely white table, with nothing over it, and it was impossible to recognize (image 3). Then we put some stuff on it, like pencils or even a keyboard, and after the software identifies them, it quickly did the same with the table (image 1-2, being the plane represented in brown). Although this may be related to limitations in the camera aspect of the device, we still think that there’s a lot of room for improvement.

Images identification only works well within strict parameters

Previously, we talked about the features that come with image recognition in ARKit 2, such as the ease of working with it or the feedback that the IDE provides in order to load identifiable images to the project. However, this is not a very flexible feature and minor changes in, say, the image’s color pattern make it difficult or impossible for the software to recognize it. Also, if the image is too simple, or it’s color range is too narrow, it becomes also very hard to identify and that is what Xcode is reporting in the image below:

Xcode's alert message about image recognition

As a solution to this problem, it would be interesting to include a trained system of machine learning in charge of recognition, letting ARKit only care about the AR part.

Bring your math skills

Creating a realistic AR experience, one that emulates real object behavior particularly if there is movement or physics involved, is not an easy task. Even though the ARKit engine takes on much of the hard work the level of math is probably beyond the typical dev comfort zone in order to implement a lifelike experience in the app.

For instance, one point that may cause some issues to implement correctly is, in the first place, converting from 2D screen/view coordinates to 3D ones in the virtual world (done through hit testing), especially considering each Node’s coordinates are given in relation to its parent’s coordinate system. Different data structures and types are used for all these coordinates, involving vector and matrix transformations to operate over them, which although efficient from a technical point of view, they are not easy to understand for someone new to this kind of domain as they involve mathematical concepts such as linear transformations and quaternions. Most of this is thankfully encapsulated in easy to call (but difficult to understand and master) function and methods.

Regarding the physics aspect of some possible AR project, several concepts are needed to configure bodies and joints correctly, as well as how they intertwine together such as mass, force, impulse, static and dynamic friction constants, etc. Even though it may be possible to set these through trial and error, some physics background is definitely helpful when working with ARKit in order to make interactions like collisions more realistic.

Finally, we found that at some points, due to jitter errors when receiving data from ARKit’s detection system, at the time of tracking the position of a certain image, for instance, or when plane anchor dimension’s change, it is at times useful to establish some error thresholds before updating or taking any action, and perhaps using linear interpolation to smooth out movements and make them more realistic. These are not that easy to implement in an effective way, making the framework as a whole quite hard to master.

ARKit hardware requirements

As it’s expected, this framework requires an iOS device with an A9 processor (iPhone 6s or later), and minimum iOS 11, while some features are only available with an A12 processor onwards.

Apart from that, in terms of phone resource consumption when executing an AR application programmed with ARKit, we noticed a significant drop in battery percentage after some minutes of use. This could be an interesting point to keep in mind when developing mobile applications in certain contexts, as battery consumption is a crucial aspect.

iOS’s only

As with many things Apple ARKit is an iOS only framework not available for Android apps. If you are serious about cross platform and feature parity you will need to bring the same capabilities to iOS & Android. Instead of having your team learn two frameworks and two ways of doing things you may want to evaluate implementing a library that supports both platforms, such as ARCore.

The Ugly about ARKit 2

Cannot identify whether the camera’s vision is hindered when spawning virtual objects

As we have said earlier in this post, spawning virtual objects is pretty easy and works fine. Problems start to appear when the camera’s vision gets hindered and that virtual object remains visible. The system is not able to realize that it shouldn’t display it as can be seen in the example below.

Screen recording of an iPhone using AR technology to spawn a virtual Squirtle

Granted, this problem is hard, and a proper implementation would probably require 2 cameras to measure depth. The ability to easily blend spawned objects with existing ones, remains a challenge not solved out of the box by the current ARKit version. Which totally breaks the illusion of the virtual object actually being there.

Poor object recognition (compile time & unreliable)

As it was mentioned before, ARKit offers the possibility of recognizing specific known objects or images in the real world. This works as long as the objects to recognize meet certain quality standards (high color contrast, no repetitive patterns, etc.) AND objects were scanned properly (from different angles, adjusting the bounding box as corresponds, etc.). Still, there are some key limitations in this, according to our testing.

The first is that both image files (with the .png extension, for instance), and 3D object data to be detected by the application, with the .arobject file extension, have to be included in compilation time as objects in a resource group to be handled by the application. This implies that all objects and images to be identified have to be known beforehand when developing the application itself. According to our investigation, ARKit offers no chance of detecting or categorizing 3D objects (eg. recognizing a certain object as a cup of coffee) other than this route, with the exception of plane surfaces. Below is shown the process of loading the information about a real object in order to allow ARKit to recognize it later:

Screen recording of the process of loading information to ARKit about a real object

The second is that ARKit uses an algorithm that compares the images captured by the device’s camera with such files, and returns a result based on how similar what is seen is to them, without any sort of artificial intelligence or learning behind the process. This means that detection suffers greatly when conditions vary, such as lighting, sight angle changes, etc. This is something to look out for if the detection of objects is crucial for the application in changing contexts, and can be improved either by improving the information ARKit is provided with, or possibly by integrating it with machine learning models trained in recognizing the desired object.

Final thoughts

These were our findings about the ARKit 2 framework after spending some days playing with it. Even though it brings interesting capabilities to mobile app development, it still feels a technology that’s in an infant state. Leaving many responsibilities up to the developer of things one would expect were solved, or at least approximated better out of the box.

Are you thinking about implementing AR in your next mobile project? Let’s get in touch!

Top 10 Mobile App Development Trends for 2019

By | Mobile Development

Each year brings its share of trends regarding technologies that push the envelope of what can be achieved in mobile app development. If you want to keep up with the mobile apps industry is important to take a look about what to expect for this year, and this post will help you with it!

We consider these to be among the top trends shaping mobile app development in 2019.

1) Artificial Intelligence: Chatbots and Voice Assistants

The presence of AI in our mobile devices is something that has been around for some years now, bringing automated assistants that can perform tasks and interact with real people, like Siri from Apple.  These assistants are improving every year by learning user behaviors in order to provide a more accurate and personalized experience. For instance, Google recently launched an AI project called Google Duplex that allows users to ask their phone’s Google Assistant to make reservations by phone call, simulating a human voice and being capable of having a fluent conversation with the person on the other side of the line.

Illustrative image of Artificial Intelligence
The same happens with chatbots, which allow businesses to automate the most frequent interactions saving humans time, and reducing cost for the enterprise. It’s estimated that by 2020, 85% of customer interactions will be handled without a human according to Gartner. Even Facebook announced that last year more than 300,000 bots were used by enterprises over Messenger, and the API to enable chatbots in Whatsapp was introduced a few months ago.

Chatbot platforms make it easier to implement chatbot based solutions, and users are becoming every day more used to text & voice based interactions with apps and software in general. It is in a way a new frontier in user interfaces, and it will affect the development of custom mobile apps. Be it as a way to enable hands off interaction with an app, or as a front-end to interact with several back-end systems through a chat based interaction, or as way for apps to respond to commands that were triggered through a voice assistant, such as Siri shortcuts.  This is why mobile developers must keep a close watch on chatbot and voice assistants.

2) Augmented Reality

Augmented Reality (AR) technologies are not a new thing but they are becoming better and more pervasive every year, reaching unprecedented levels. Mobile games like Pokémon GO made this so popular that thousands of people were interacting with virtual objects and creatures that don’t exist but were displayed over the screen thanks to AR. However, applications for this technology can go even further in business critical fields such as: security, health, training, industrial maintenance and more.

Bulding model shown on an iPad by using AR technology

For instance, police officers in China are starting to wear smart glasses that allow them to identify suspects by face-recognition systems and even spot criminals in the crowd. Although they are still experimenting with it and working to exploit AR in the fight against crime. It is an interesting use case of applying AR in society’s matters and no doubt it will occupy an increasingly important place because, according to Statista, the number of AR glasses shipments worldwide will reach around 22.8 million units by 2022, and its market size is forecast to hit more than 20 billion U.S. dollars at the end of this year.

Apple has recently introduced the ARkit to foster augmented reality experiences in custom apps, based on the camera  capabilities of both the front and rear camera. Enabling iOS developers to add elements to the camera display in a way that makes those elements appear to inhabit the real world. This has huge applications in gaming, but also in enterprise information systems where real world objects such as business cards can be augmented with animated metadata on the fly.

3) The Internet of Things

Nowadays, almost every gadget or household appliance that you can think of is being built with Internet access. And not only for checking Red Sox’s last game score in the microwave screen while you heat the waffles, but for connecting with the rest of the house and electronic devices in order to become a unified system that can be managed by your smartphone. Jokes aside, the called “smart homes” are starting to grow in number and becoming a game-changing topic for mobile development.

"Smart Home" representation with the use of an iPad

This is because there is no better way to control your house behavior than by using your phone. Imagine reaching (or leaving) your home and being able to unlock doors, open windows and turn on lights with just one touch. Those things are possible now and keep being improved to allow domestic appliances perform more complex tasks, managing them with mobile apps such as Apple’s “Home”. In fact, according to IDC, investment in IoT is expected to surpass the $1 trillion mark by 2022, and the development of mobile apps for that purpose goes hand in hand.

In this sense Apple has introduced the HomeKit library in the iOS SDK bridging the gap between mobile developers and the sea of iOT protocols, and smart devices and gadgets out there.

4) Mobile Wallets

There was a time when we had to pay for everything in cash. Then credit and debit cards appeared so we didn’t need to carry a lot of money bills in our wallets. But there is a common denominator here, the wallet. That thing that we usually have in our pockets didn’t change. Until now.  The wallet of the future will be your own mobile phone, and that future is starting to become present.

An android phone being used in a payment terminal

Mobile payment systems such as Apple Pay are growing in usage and popularity. According to eMarketer, by the end of this year about 36% of the worldwide mobile users will use a mobile payment app. Maybe one of the most advanced markets in this sense is China where not only upper and middles classes are using mobile wallets for their daily purchases, but word is that even some beggars use the famous WeChat app to ask for money.

In this context mobile developers must be ready to add mobile payment capabilities to their toolbox. There’s Apple Pay, but that’s just one in a growing market of mobile ready payment gateways.  Also as in every other topic when something becomes more and more popular, related features and capabilities increase in importance as well. For example, elements like biometric, and multi factor user authentication in apps should be something to consider for mobile app developers.

5) Biometric Authentication in Mobile Devices

Using your  body to prove that you are actually yourself is a common feature of new mobile devices. In fact, several researches prove that more than 60% of smartphone shipments come with fingerprint scanning capabilities, and that number keeps growing. This kind of authentication technology is being improved every year, developing a faster and precise scanner that is now even built below the screen.

Fingerprint scanner

Biometric authentication in mobile devices is now a mainstream. Not only used to unlock your phone but also to log into your bank account or any other sensitive digital space where you expect to have your data protected. Obviously, like any other security measure in the mobile world, it has its flaws, but biometrics combined with classic authentication methods represent state of the art authentication security. Mobile developers must be ready to include technologies such as: FaceID, or TouchID capabilities to their  mobile apps.

6) Accelerated Mobile Pages (AMP) and Progressive Web Applications (PWA)

Making a fast and user-friendly website that performs well in any device (especially in mobiles) is one of the main goals for developers. People leave sites that take too much time to load, and the same happens with applications. AMP and PWA bring new ways to tackle and old problem.

Programming code shown on a MacBook

AMP (Accelerated Mobile Pages) is an open-source library that provides a way to boost the performance of a web page and to improve its load time. It was introduced some years ago and gained popularity among web development because of its effectiveness. On the other hand, Progressive Web Applications are web apps that behave just like a mobile native app in order to create a reliable, fast and engaging experience for users. The advantage of this technology is a cheaper development cost comparing to developing a traditional native application, but with closer results.

AMP and PWA can be integrated together to take the best of both worlds, and that is why mobile devs are working with them having in mind that most of the web browsing occurs in a mobile device. Not only they enhance user engagement in your website, but also improve your visibility over search engines getting more traffic as a result. Because of this, both technologies form an important trend in mobile app development for 2019 that you shouldn’t miss.

7) 5G Technology

The next generation of cellular networks is hitting the door, and it is expected to reach the market by 2020. Experimental tests and researches hold that 5G is faster, cheaper and more stable than 4G, consuming even less energy. This is why there are already mobile phones being developed with this brand-new technology in order to be prepared for its arrival, such as the Samsung Galaxy or the Moto Z3.

Illustrative image of the 5G network

Even though everything is still too recent, mobile apps and the device as well needs to fit the 5G requirements, because we are talking about the network of the future. According to Statista, the number of 5G connections is estimated to reach between 20 million and 100 million by 2021. And a mobile device can lose quickly its user-engagement if it is not suited for that new technology meaning it won’t have a connection speed as fast as the competence.

So, what does 5G mean for mobile developers? Well, among other things: improved content, and specially video based experiences, including 4k video streaming, but also bandwidth to better handle heavy VR & AR 3D objects. Not to mention app bundle size becoming less of an issue, and more users willing to work on the go as the gap between mobile and wifi connectivity becomes less noticeable.

8) Foldable Displays

The breach between phones and tablets is merging into a new category; the foldable phones. These ones are starting to become  known and the first models are expected to appear this year, like Samsung’s Galaxy Fold and Huawei Mate X. DSCC (Display Supply Chain Consultants) forecast that in 2019, around 1.65 million foldable OLED panels will be shipped worldwide, reaching 60 million units by 2022 what shows a clearly fast grow.

A photo of the Huawei Mate X

For the time being, this is only a thing in the Android side of things, but it will have implications when developing user interfaces. Mobile apps will probably need to be optimized for that instant change of screen size between the folded version and the unfolded one. It will introduce yet more complexity and dispersion to the already crowded space of different screen sizes in the Android world, maybe opening more opportunities for players that automate the generation of user interfaces in apps.

9) Beacons & Nearby Networking

Even though the Beacons technology has been established for some time now, it didn’t reach its full potential yet. Beacons are devices that emit BLE (Bluetooth Low Energy) radio signals that smartphones can see and in most cases respond to. For example, stores can take advantage of this by sending day offers or information to customers that walk into the store, through their own app.

Illustrative image of a smartphone receiving Bluetooth waves.

That’s not the only thing BLE could work for. In fact, we have created ourselves a nearby messaging app based on BLE, and it’s also heavily used in mobile payments scenarios. It remains to be seen whether BLE and beacons in general will take off in 2019.

10) Mature hybrid mobile app development frameworks

The iTunes app store has turned 11 this year. After an initial cambrian period of exploration the market has settled several years ago on two main mobile platforms, iOS and Android. These platforms have matured and in turn this has enabled a number of frameworks such as: React-Native, Flutter, Xamarin, Kotlin-Native to evolve over the years. The problem of managing and keeping up with multiple code bases for one mobile app is a tough problem. Any one who has struggled to achieve and sustain feature parity providing a common user experience among platforms knows this.

Image of an Android phone showing its apps.

This is not a breaking trend but more like a creep, or a steady rise of what represents a viable alternative to native development. The space of user experiences only achievable through native development is slowly but steadily becoming thinner over the years.  There will probably always be experiences that only native mobile development will be able to achieve, but experience in some hybrid development framework should be part of the toolbox of any mobile app developer this year. Particularly for developing MVPs, or any early stage product release where product market fit is not yet assured. In these contexts hybrid development frameworks provide an increasingly solid path to save money, time, and headaches.

Final thoughts

The list of trends mentioned in this post is based on research and the opinion of our mobile team, but it’s not by any means extensive. Other trends did not make the cut, such as instant apps, and blockchain in mobile, but what do you think? What are your thoughts about this year trends? Leave a comment below and tell us!

Generating REST web services from PostgreSQL tables

By | Uncategorized

We’ve just released another open source project in github, a humble utility we’ve been using to generate and maintain data intensive rest endpoints that have a CRUD behavior on database tables.

It’s implemented in Java, and it generates:

  • Java Beans for tables.
  • JDBC based Data Access Objects to retrieve and execute commands on the PostgreSQL database.
  • Jersey 2.0 endpoint classes for each table.

The goal was to automate the creation of straight forward endpoints, and to separate data structures, and queries affected by changes in the database in base classes meant to be inherited, and updated regularly as the database evolves. So that, every time you make a change in the database, you just run the generation process and it updates all your related code.

All the generation is driven by templates files, with some ugly tokens for dynamically replacing content and at times for required pre processing of conditional logic.

The generation is based on metadata queried from the PostgreSQL database, but it could be extended to support other relational sources.

It generates Java code, but we are thinking about moving to node.js REST endpoint generation, in the near future. Suggestions and contributions in this sense are welcomed.

Check it out, and please do provide feedback!

A slack alternative that runs on a Raspberry Pi

By | Uncategorized

We’ve made available a docker image of our enterprise chat, task manager (and slack alternative) Kezmo. We used a Raspberry Pi 3 as our infrastructure benchmark for a fresh installation ready to support a small team and it works 🙂

So, what’s Kezmo?

Kezmo is an enterprise chat and team collaboration platform that can be customized and integrated to any system in your organization. It’s based on a small set of core concepts, which are: work spaces, conversations, containers, and container items. A workspace is a pocket of context to collaborate around any given subject. It can be a team, a project, an short term initiative, a client, you name it. Members can be invited to workspaces by it’s creator, where they can exchange conversations, documents, notes, tasks which are stored in workspace containers. The underlying design goal was to provide an intelligent way to store and categorize shared content by type,  attempting to overcome chat archeology whenever you need to go back and recall what was the agreement, link, or file shared at any time.

You can try the vanilla version from the cloud, or you can run it on premises. We implement white label versions of Kezmo, with custom content types, forms, and automations to integrate it into domain specific systems, in health, finance, and construction. If this could interest you let us know.

Where can I get it?

You can try the vanilla version from the cloud by signing up here, or your can try it in your own premises with the docker image we’ve recently released. We would really appreciate any feedback on the setup process.

How to build your customer support chatbot with DialogFlow

By | Artificial Intelligence, Chatbot, Machine Learning

We have recently released our support chatbot in Kezmo (our own @kezmobot), implemented with DialogFlow (former API.AI), and this blog post details some of the lessons we learned building it.

Kezmo is a task management & messaging platform for organizations that require an agile internal and external communication.  The product includes a support space where users can chat with our team. Up until recently support was handled by a subset of the whole dev team. While this has been great as a way to get insight into our product’s usage it’s also true that pareto holds and 80% of the users tend to ask about the same things, hitting 20% of our knowledge base. Also, answering customer questions during weekend nights is not fun after a while.

So we set out to build our own customer support chatbot.

Why API.AI?

If you are about to build a chatbot you’ll learn there are number of alternatives out there provided by the major players in technology, and then some more. Among them: API.AI from Google (recently acquired), Microsoft Bot Framework, IBM Watson.

API.AI was recommended to us by our good friends at the Tryolabs team. When evaluating alternatives such as the Microsoft Bot Framework we realized that API.AI requires less code plumbing, and overall proposes a more simple approach (higher limits on number of intents, no need for handlers on intent callbacks, no multiple cognitive services and endpoints). Plus API.AI is free, right? After exchanging emails with Google’s API.AI sales team, I’ve learned that:

Our platform is free to use as provided but we do restrict requests per minute to 180 on average. Google Cloud will add paid tiers later in 2017 with new features that align with enterprise requirements and allow for increased request rates.

They said the free version isn’t going away. We’ll see about that. So, for now free means you can make up to 180 queries to API.AI per minute, so you get about two per second. For the time being this works for us.

Getting started with API.AI

In API.AI chatbots are defined as agents. In API.AI you define an agent based on intents, which are the conversations interests it can handle. For each intent you define a number of phrases, texts, used to train the model. So that whenever a similar phrase is found that intent is matched. In our experience, in API.AI you should think about loading around 20 sample sentences for each intent for the model to behave reasonably good.

You connect to your agent using any of the supported out-of-the-box channels, such as Facebook Messenger, or Skype, or you can code your own client using any of the supported libraries to connect with API.AI. Since we wanted our bot to run on Kezmo, our own messaging platform, we had to program the integration.

When you create an agent you can base off existing sample data which preloads the agents with typical intents, or you can import prebuilt agents, which also preload intents into the new agent.

We wanted our bot to handle basic small talk, so we imported the small talk pre built agent, which loaded 84 intents. The imported intents do not include preloaded answers. So expect to spend some hours loading content in order to enable casual conversation skills.

Then we moved on to load our FAQ knowledge base as intent responses. In order to do this we combed all our support conversations for the past six months to identify intent patterns, and mine typical phrases used to express intents. This took days. We ended up with more than +500 sentences expressing intents.

Output formatting of the chatbot responses in API.AI

For a first version we loaded the answers directly to API.AI, I know we could have implemented an action, and a fulfillment to retrieve them from some endpoint but not for version one (make it work, make it right, make it scale).

The first issue we faced was supporting line breaks in the response texts loaded in API.AI. For some reason this gets lost in the client call, so we had to include markup (\n) and a basic post processing code for the API.AI calls to replace \n occurrences for actual line breaks in our messaging platform.

Implementing the bot human handoff with API.AI

Kezmobot is a customer support agent meant to act as first level support, so we need it to be able to escalate questions in a number of scenarios, including when the user explicitly requests to talk to a real person. Our support channel is a group chat, which means it’s key that the bot remains quiet once the human agent takes over the conversation, otherwise you get the nightmare scenario where the chatbot answers both the client and the human support agent.

Transferring chat to a human agent is not something  API.AI supports out of the box. You need to add some client code in order to implement this sort of behavior.

In our case we relied on a hand off context, that whenever returned as output of an API.AI call the client code would simply sleep the bot for 10 minutes, for the current client conversation (it quickly became clear it needed to be more like 12 hours).

Since we wanted to achieve a natural looking transition we created an intent that would match phrases such as: @kezmobot I’ll take over from here. This intent outputs the QUIET-MODE context notifying the client code to stop passing calls for that conversation, and that’s how kezmobot goes silent.

Handling sentences with multiple intents with API.AI

After combing our support log we realized humans have a tendency to pack multiple intents in one sentence. Interactions like the following are common:

Hello, how are you doing? I wanted to ask about X feature, how does it work? By the way I love your product, thanks!

Here we have at least, 3 intents, like:

  • Opening Greeting
  • Feature question
  • Closing thank you

These type of sentences can lead to a mismatch of intents. The model may match any of the three intents, leaving two unanswered. What we did was implement a very basic pre processing of the text we send API.AI to recognize stop tokens, used to split the sentence in multiple calls. For instance, we would split the sentence on occurrences of substrings such as:  . ! ? which would output something like the following:

  • Hi
  • You can use feature X in this or that way.
  • Glad you like our product! (would be great if you could post a good review on the store!)

That’s half of the solution. The other half is adding in our welcome intent wording to let the user know our bot works best with short simple phrases 😉

Measuring your chatbot accuracy

We loaded a CSV file with two columns. On the left column a sample text from our support logs. On the right column the expected intent for each phrase. Then we ran a process that would call API.AI using each line in the CSV file, and check the output against the expected intent. The difference between expected, and actual was used to calculate the % of accuracy.

One caveat is that you have to sleep your calls to API.AI to avoid going beyond the allowed throughput for the free service.

Between executions we tuned the ML Settings for the agent. Basically we played with different numbers for the ML Classification Threshold. We settled (for now) on a 0.5 value, which worked best for us to avoid false positives.

Implementing a chatbot that can handle multiple languages with API.AI

A big part of Kezmo’s user base is from spanish speaking countries, so it was key for us to support multiple languages.

They way to implement this is using multiple agents, one trained for each language.

Currently we have released the bot for spanish, and we are working on tunning the english version of it.

The good thing is that with our approach the client code does not care about the spoken language, or even the intents at play. We rely only on existing contexts, that must be shared among languages. What sets them up is transparent for the client code.

Chatbot analytics on API.AI

For the time being API.AI analytics are pretty basic. It’s rumored that they will open Chatbase to the masses some time. In the meantime log your own stats in your database.

Closing Thoughts

API.AI is adding features on a monthly basis. Business model is still not rock solid. But it’s simple, and gets the job done. It provides a good balance between expressive power and simplicity of the whole developer experience.

Real world usage of our chatbot is not like our human interactions, and now we are working on a second round of tuning after real interactions people are having with our bot.

If you want us to check your support process to see if we can implement a chatbot for it drop us a line, we’d be happy to help you!

Salvar

Salvar

Salvar

Salvar

How can you make UITableView / UICollectionView with dynamic headers

By | iOS, Mobile Development

Scroll based interfaces are everywhere. It has been the de facto overflow solution since we saw the first modern user interface, and of course it is widely used everyday on our mobile devices today.

But even though scrolls are very common, we have all seen apps spicing the scrolling experience up in different ways, and as mobile developers we want the best experience to delight our users too.

Having this in mind, I decided to create and share a simple UI component that hopefully will help you create better experiences as well.

Introducing the OLSDynamicHeaderViewController

OLSDynamicHeaderViewController is a simple iOS UI component that allows the most common scroll based UIs to have a header that can be easily animated along with user scrolling, as simple as that! So…

How do I get it?

The code is published on GitHub as you expect, with a sample project showing the basic capabilities. You can either use it through CocoaPods or you can also copy the files directly to your project, old school style.

How do I use it?

First, make your UIViewController to inherit either one of the helpers OLSDynamicHeaderTableViewController or OLSDynamicHeaderCollectionViewController.

 

Now, you need to provide a header view to the component, and for that, you need to implement the headerViewInstance() method and provide an instance of a OLSDynamicHeaderView subclass. This view must conform to OLSDynamicHeaderViewProtocol that consists in the following methods:

 

  • maxHeight() returns the maximum height allowed for this view.
  • minHeight() returns the minimum height allowed for this view.
  • resize(withProgress:) will be called when the user scrolls and your header needs to animate. `progress` goes from 0 to 1, 0 min height reached, 1 max height reached. This is probably the method you will be working the most to pull out a nice animation 😀
  • overflow(withPoints:) will be called when the user scrolls further the max height. `points` goes from 0 to N.

Cool! … anything else?

  • Note you can also inherit from OLSDynamicHeaderViewController directly if you do not want to use a UITableView or UICollectionView. The code of the corresponding helpers are there for you as reference of a base implementation 👍🏼
  • The sample code has a SampleHeaderView class that you can use (of course), but also have some basic math you may find handy for your header ideas!

Happy coding, cheers 🍻

Salvar

Will chatbots really kill the app store?

By | enterprise, Enterprise Collaboration
bot bender smokes stooggies

chatbot hype?

I recently came across the following article: Chatbots will soon kill the app store. In a nutshell the author states:

Messenger apps are huge already

Granted, we also believe this, that’s why we are investing in building Kezmo, a customizable messaging platform that can be setup on premises, so Enterprises can get the best of both worlds.

Chatbots are going to be huge (yyuuuuge)

We believe that chatbots will play a major role in the future of human computer interactions, and yes we think they are going to act as the trojan horse for the arrival of artificial intelligence. Still, the hype around the subject does not quite match the fundamentals. I don’t on a daily basis use or interact with any chatbot. The people I know don’t. Every chatbot interaction I’ve had has been in the context of toying with a new technology. I have not adopted it in any scenario yet, and I don’t know anyone who has. So, while I’m bullish on bots, and Kezmo will serve as the foundation for the deployments of bots either the technology is not quite there yet, or otherwise

The app store user experience is a mess

Indeed, to our dismay the app store has proven to be zero traction marketing channel. The fact that the majority of mobile apps are zombies is something we’ve experienced first hand.

Today 83% of the apps in the App Store are zombies

It doesn’t seem that far ago when free apps were able to generate hundreds of downloads per day just by the mere fact that they were free, and the app store was new. This dynamic is dead. If you are publishing anything to the app store, make sure to generate traction for it independently, relying only on the app store for marketing an app is not an option anymore, no matter how much app store search optimization time you put in.

Closing thought: are we comparing apples to oranges?

We can all agree the app store user experience is a mess, and both Apple and Google need to make changes asap to inject some life back in the app ecosystem. On the other hand chatbots seem like a promising investment despite imho the lack of true traction yet. To what extent are these two phenomenons related? Are we comparing apple to oranges?

What to expect from TechCrunch Disrupt Startup Alley?

By | Kezmo

So you are thinking about going to TechCrunch Disrupt, maybe even exhibit at the Startup Alley and you are wondering… is it worth it for my startup?

We’ve just returned from our TC Disrupt trip to San Francisco last week. We were there to launch Kezmo, our enterprise chat and team collaboration tool. We had a table at the startup alley, as part of the Uruguay pavilion, so we can tell you exactly what to expect.

3V4B1083

What to expect from TC Disrupt

  • I see startup people. People, thousands of them. This 2016 edition had somewhere between 3000 and 4000 attendees. Most of them startupers. Fellow startup founders, desperate to be noticed, which sets the stage for a hard competition for attention.  Beyond the numbers it’s important to stress the fact that you see people from all around the globe. TC Disrupt has country pavilions, as part of the Startup Alley which adds to the cosmopolitan vibe. It’s good if you want to get a feel of how your product resonates with different cultural backgrounds.
  • Venture/Vulture Capital. You will talk to some VCs representatives, and some VCs wannabes. The latter are dangerous as they can be tire kickers and will suck time and focus out of your brains. Be warned.
  • Tech Journalists. If you exhibit you get a list with a couple of hundred journalists that will be there. You can email/spam them with very low chances of getting a response back. You will get to talk to some of them during the event…but unless you either know somebody who knows somebody and can introduce you to a journalist, or if your startup is somehow related to one of the darling topics of the moment (this year it was all about AI & Virtual Reality) getting coverage will be pretty hard. So low expectations there, and start moving those connections, or talking to those PR agencies.
  • Students. Hackathon winners, university students, MBA graduates, you name it.  They are not the majority but you will come across them. They will listen to your pitch with a lot of attention (maybe too much?) only to let you know later that they have just finished this career/course and are looking for a job. But hey, if you are running a startup outside the US. and are looking to expand your operations in that country it can be an opportunity, right?
  • Talks. If you follow the startup scene, you will notice that people you admire will be giving some cool talk. Unfortunately you won’t have time to attend to most of them. Unless there’s three of you tending to your booth, and you can just leave. For insance in this edition Neil Stephenson was signing books, and I loved Seveneves, but I really couldn’t make it. It was rush hour at the booth and before I noticed they day had finished.
  • Business Development Opportunities. Among the army of startups presenting you will come across some interesting companies. Competitors for once, but also startups whose business could be a good complement to yours, and it’s in these cases that you have an opportunity to discuss some kind of arrangement that may server both parties.
  • Interesting contacts. I came back with +50 business cards, of various degrees of relevance. There are probably 5 of them that could prove to be valuable contacts in the future. From other tradeshow experiences I was expecting to come back with more like 200 hundred cards. Maybe it was just poor hustling on my side, but those are the figures. On the other hand I know I delivered hundreds of my own cards, and was left without any by the second day.
  • Trends. You don’t have to be go to Disrupt to stay on top of trends, but it never hurts to be there and get a feel for what’s trending in the startup scene.

What NOT to expect from TC Disrupt

  • End users. Unless your end users are specifically startups, and startup founders, but even in that case, they will be probably too busy to pay meaningful attention to you at least during the event. If you expect people to hear about your product and register that same night you will probably be disappointed.
  • Enterprise. Large corporations do not have a big presence at Disrupt. With some exceptions that send R&D representatives to spot trends, most large corporations are just not there. So if you plan to connect with these type of companies, or if you are looking to connect with CIOs or CTOs you should start looking somewhere else, maybe the Gartner events?
  • Secured Coverage. In my opinion TC could do a better job at providing exhibitors better coverage. As an exhibitor you get listed in a lousy dynamically generated page that’s very hard to find & navigate. It does not appear in search engines, and where you are listed along with hundreds of startups within a collapsible accordion (I dare you to find Kezmo). For days I was expecting to have some kind of public url I could use in tweets or blog posts, if only to make a statement with local investors that our presence at Disrupt was real but what they provided arrived late and was pretty poor.

Final thoughts

We tried to make the most of the event by launching Kezmo to the public at Disrupt. To some extent this did provide some news coverage,  from media we were not expecting in regions we did not foresee.

Our previous experiences in tradeshows and conferences came from the Microsoft world, in SharePoint events, such as SPTechCon. In comparison that type of events are more enterprise oriented, have more CIOs & CTOs attending and offer a more focused audience for the type of products we develop.

In a couple of weeks I will know for sure if the event really paid off. For now it’s a close call.

Should you go? Sure, provided you have the right expectations.

 

Save

Save

Kezmo is finally available to the public

By | Uncategorized

We are proud to announce that Kezmo is finally available to the public.

Kezmo is an enterprise chat and collaboration tool for managing projects & teams. We believe one of the main problems with most enterprise chat tools is that everything is a message and after a couple of days of having a conversation with your team you forget what were the agreements, actionables and who owns what.  Kezmo is a slack alternative that enables enterprise conversations while providing means to easily transition to more actionable structures such as: tasks, issues, notes, etc, that have their own life cycle and tools to interact and manage them independently.

You can get started by registering online, and you can get the mobile clients available in the iOS app store, and in the Android play store.

We’d love to have your feedback.