Jun 12
1

What does 5G mean for Mobile Apps?

By | 5G

An illustrative image of the world connected.The new generation of cellular network technologies is hitting the door and over time we are hearing more and more about it, especially in the news. Known as 5G, the wireless network of the future, this new technology has started to grow in popularity and a lot has been said about things like its improvements in comparison to 4G-LTE. But what happens in the mobile app development field? What does 5G mean for mobile apps?

To begin with and for those who are not so acquainted with the subject, 5G is the fifth generation of wireless technologies that promises faster and more reliable connections.

Many companies such as SK Telecom in South Korea and Verizon in the US have already started deploying 5G this 2019 in their respective countries, offering connectivity plans to customers in a certain number of locations, a number that keeps growing with time. In other places like Latin America, the Uruguayan state-owned operator ANTEL in partnership with Nokia successfully completed the installation of a 5G commercial network, making Uruguay the first country in the entire region to have one.

Now, let’s see how this arrival of 5G could impact mobile applications and their development.

 

Higher speed connections

Several tests have been conducted in regards to the internet connection speed that 5G delivers and some results are very interesting. In fact Verizon’s 5G network managed to reach 1 Gigabit speeds, whereas with standard LTE connection this is not easy to achieve or just unachievable. That means faster times loading and downloading apps no matter their size, enhanced web browsing, streaming videos with no lag, and better performances in applications that need access to an internet connection.

Below is a short speed test video from a user in Bloomington, Minnesota, connected to the Verizon network:

Imagine being about to board a plane, with 10 hours of flight ahead, and realizing you have forgotten to download your favorite episodes from Netflix’s Stranger Things. This would be a problem with a 4G connection, right? You will need a couple of minutes (in best scenarios) in order to have them available offline. Fortunately, with 5G top speeds, this would be fixed in just seconds as you can download every episode or film you want instantly.

 

Near-zero latency

Every developer dreams with apps that respond in a flash, meaning also quicker loading times for every action.

This brings improvements in AR/VR implementations where latency is a key factor in order to create the feeling of being in a realistic world. But it’s also a game-changing upgrade for user experiences in apps that demand a huge retrieval of data, or processing from remote servers which clearly reduces the boundaries of mobile hardware, especially in high-demanding tasks like using Machine Learning capabilities.

Going to the numbers, 5G targets latencies of 1ms or less, a milestone that would be really appreciated, for instance, in remote surgeries where every millisecond of delay counts.

 

Better file transfers

As a result of being both improved the connection speed and its latency, then sharing files between mobile devices would get better too, not only in terms of velocity but also in making this sharing more reliable with low chances of losing data in the process.

This means that more people would be willing to work on-the-go or remotely without the need of having an established WiFi connection, exchanging files large or small with partners and clients in an instant, something that makes distances even shorter than now.

Yes, you could receive a huge 5 GB file from someone on the other side of the world in less than twelve parsecs. Han Solo would be proud. Jokes aside, 5G is expected to be 20 times faster than 4G in terms of peak speed, getting quicker file transfers as an outcome. Impressive.

 

IoT improvements 

Definitely the real winner here is the Internet of Things field. Why? Because 5G promises to support almost 1 million connected devices per square kilometer, whereas 4G can only handle up to 100,000.

This brings the possibility of having, let’s say, everything connected to the same network simultaneously in the same space, including smartphones, wearables, cars, and even the city infrastructure, sharing data with each other.

The improvements of IoT are not only reduced to smart-houses or rooms (clearly these ones would benefit a lot), now you could be able to interact with the entire neighborhood by using mobile apps, like asking for a self-driving car (also connected with traffic lights and others of its kind) for transportation. If the Internet of Things is now a trending topic for mobile development, imagine how important it would be with the arrival of 5G features.

 

Enhanced performance and battery consumption

Oh, battery consumption, what are we going to do with you? Every time a new technology is implemented in mobile devices, like ARKit or ML, and no matter how good it looks, the battery seems to drain like if there was no tomorrow.

But this seems not to be the case, for our luck. If less computing is needed inside the phone terminal and the 5G network has a wider reach and stability, then mobile devices would spend fewer hardware resources, having in mind that this new network will be probably better optimized.

The same happens with performance. Connectivity would be improved in remote areas, as well as GPS and location features. Low latencies and faster speeds mean better response times for online mobile applications. Also gaming would take benefit from those things as you will be able to play high-demanding games on your phone by streaming them from servers, which is commonly known as cloud gaming. Therefore, you are not needed to own a high-end device with expensive hardware (in most cases there is no mobile phone capable of running those games) as all the hard job is done by the server. This is something already being worked on and there are several platforms offering it like Google Stadia (set to launch in November 2019). No doubts 5G will boost them.

 

Conclusion

5G is not a fully matured technology yet and we only have few experimental tests out there, so results can vary in real conditions. Even though we listed some improvements for mobile apps thanks to the arrival of 5G (but our list is not extensive at all), there are still a few challenges it will need to pass through.

Things like security and privacy (a hot topic in our days) could be compromised as more devices can be connected in the same network and more data would be shared. However, it’s pretty clear that it’s the cellular network of the future and we’ll see a lot more of 5G in these months and years to come.

We’re always looking forward to new emerging technologies and our mobile app development team is getting ready for them. What about you? What do you think about 5G arrival? Leave a comment below and tell us!

 

May 28
1

Getting started with Machine Learning in Mobile App Development

By | Machine Learning

An image of Darth Vader with a whiteboard behind that says "Machine Learning"Machine Learning is, at the time of writing, a hot topic in the tech world and especially among software developers. Its integration in mobile apps is now easier to reach thanks to the launch of Core ML (exclusively for iOS) first in 2017 and Google’s ML Kit one year later. We spent some time researching both platforms and their features so that if you are wondering how do they work, this is what it looks like. Let’s put some background first.

For those who don’t know, Machine Learning is considered a subset of Artificial Intelligence that constitutes a grouping of algorithms that allow systems to learn and improve from data without being explicitly programmed. It focuses on prediction operations based on known properties so that its methodology could be summarized in “based on what I know, what you’re giving me is this (result), and I’m x% sure.” In order to perform these predictions, it’s use what is called a Machine Learning Model which is basically the result of training the learning algorithm with sample data. Now, let’s see how mobile big players like Google and Apple help developers to get a hand of this tech on their platforms.

 

Core ML

Core ML is Apple’s framework for Machine Learning integration in iOS apps. It is built on top of already existing low-level technologies like Accelerate, BNNS, as well as Metal Performance Shaders. Built above it, there are higher-level frameworks that work with specific domain tasks like Vision for image analysis (faces, barcodes, and objects) and live recognitions, Natural Language for text and speech recognition (supports already existing Natural Language processing models, NLP’s), and GamePlayKit (AI, pathfinding, agent behavior) for evaluating decision trees.

The models that Apple provides out of the box are several and ready to be used (also free), from detecting the dominant objects present in an image to detecting the scene of an image from a finite number of categories such as bedroom, forest, coast, and more. These models are stored and work on the client side, so that results are near real-time as there is no network latency and also a server is not needed for them to run. In terms of security, the fact that there is no need of connecting to a remote server means that none of the inputs or outputs related by the model used are sent away from the device and, in consequence, no one can externally grab that information.
Another feature of those models is that they’re optimized for on-device performance, which minimizes memory and power consumption. Unfortunately, having everything local clearly brings an increase in the size of the application (there’s a workaround for the size problem using CloudKit, storing the models to download them on-demand and allowing updates without the need of updating the app itself).

Moving on with how to put it to work and as we are going to show below, using a model to predict something is just a matter of a few lines of code.

Getting started with Core ML

The first step is to download the model, choosing between the ones from Apple’s Developer site, or using a customized one from any other ML toolbox such as TensorFlow or Keras (in order for them to work you need to convert them into Core ML format “.mlmodel”, something easy to achieve as Apple provides tools to do it ). A fully open source API to export and create models is available, with support for standard model formats that can be translated to the one used in iOS. There are also tools that allow you to reduce the size of big size models (some scenarios can go from 500MB down to 5MB).
The next step is to import the model into your Xcode project: this automatically generates all the necessary Interfaces to interact with the model, ready to be used as soon as you instantiate it and call the prediction(..) method.

For instance, if you’d like to work with every model out of the box, the code would be:

 

What you are doing here is using the model with the input values that it needs, being images in this case and receiving all the results (Output) that the model detected with the corresponding input. We built a basic app in order to try them with the ease of taking a picture and then analyze it with the model of choice, and here is the result:

As you select a model the objects are listed below from the one that has more probability of being in the image, to the one that has less. The percentage of probability and its accuracy is something that ML developers work hard on, and this varies with each model. Let’s move on now with Google’s framework.

 

ML Kit

ML Kit is a mobile SDK that Google provides to allow Machine Learning integration in mobile apps. It’s a Firebase based framework, optimized for mobile that works both in iOS and Android with all of its features. Something to highlight about this framework is that includes a very broad set of out of the box API’s that already solve the most common problems in Machine Learning. These models are handled internally by the default dependency manager of Android apps called Gradle, with no need to worry about getting them and they keep being updated automatically over time. Here we have a huge difference with Core ML where in order to maintain a model up to date you are required to upgrade the entire application, manually. ML Kit also supports third-party models, but the available sources are limited to TensorFlow Lite models for now.

As soon as we get into the Firebase platform we came across a friendly interface where the available models are listed, being Text Recognition, Face Recognition, Image Labeling, Barcode Scanning, Emblematic Places Recognition, Language Recognition, and Smart Replies. Besides, in Google IO 2019 conference which took place from May 7 to May 9, it was announced the implementation of AutoML Vision Edge in ML Kit. This new feature allows devs to train their own custom Machine Learning models with ease, in regards to the image labeling field.

One thing to notice about ML Kit models is that some of them are available to work in the cloud improving drastically the accuracy and the number of categories to classify from (this service is free only for a small number of requests), others exclusively allow on-device processing whereas Text Recognition and Image Labeling are the only ones that work both cloud and client-side.

Getting started with ML Kit

To run a model using ML Kit (we’re going to use Android as an example, but the process is quite similar for iOS) we need to set up a project in the Firebase platform and connect it to your chosen app. Once is everything set, the next step is to add the Firebase SDK dependency manager (Gradle) into the application itself. This will vary depending on the specific model/feature you wish to use, but they all share the same package base. Having this done, it just a matter of including the Model class, then configuring it with the options needed, and calling the processImage method as its shown in the code below:

 

In order to test the models that ML Kit provides, we worked with a demo project that comes out of the box. We recorded the same objects used above with Core ML focusing on the Image Labeling model as it’s the one that looks more similar to the way that Apple’s framework recognizes objects but identifying them live and showing the results on the screen. Because it is a live recording, those results are not fixed being constantly updated, and this is how it looks like:

Here is a closer look: A screenshot of a ML Kit Demo project.

Final thoughts

In this post, we focused on object detection with image inputs in both Core ML and ML Kit. However, Machine Learning can be applied to other types of entries such as texts, numbers or even sounds, returning also the same variety of outputs. This brings interesting possibilities like recognizing credit cards for fast payments or decoding messages for chatbots and AI implementations.

As from our experience, using an existing ML model in both platforms requires basic mobile programming knowledge and integration time. Both platforms put a lot of effort encapsulating the mathematical complexity of the model’s algorithms and providing out of the box models to solve the most common problems. But if you need to create a custom model fitted for very specific needs, then the required amount of knowledge both in maths and coding escalates quickly.

Are you thinking of implementing Machine Learning in your next mobile project? We would love to help!

Apr 16
0

Introducing IdeaSource: Open Source platform for open innovation

By | IdeaSource

We are pleased to announce that we’ve released our latest pet project, called IdeaSource. It’s a very basic MVP (stressing the minimum part) of a cross-platform app to enable crowdsourcing of ideas, that we used internally as a sandbox to research React Native development front to back.

IdeaSource home-page.

Why IdeaSource?

Coming up with good ideas can be hard, especially after a while when echo chambers tend to appear in teams, groupthink takes over and creative solutions are left out of the conversation. We wished to introduce an easy way to share and manage ideas generated by a larger pool of people than those directly tasked with coming up with a solution, providing mechanisms for everyone to vote to sort the proposed ideas. It’s about opening the discussion an enabling everyone to participate in a creative process, both in the generation of ideas, and in their sifting/sorting by means of likes.

How does it work?

Ideas are gathered around challenges, which can be problems, aspects or topics a group or organization may need to fix or improve.  A challenge can be anything ranging from the next feature/product to invest in, to what could be the new perk in the office, to creative ways to address pollution in a municipality.

In order to create a challenge you must enter a description, upload an image and set a deadline. Challenges can be either private or public. Private challenges are only available to users registered using the same corporate email domain, while open challenges are open for every user in IdeaSource. We may review this in the future, but for v1 this is the way it works.

Once a challenge is submitted, everyone can start contributing ideas, and votes for ideas already shared.

Main features

  • Challenge creation and management: Create open or closed challenges scoped only to users that share your email domain. Screenshot of the creation of challenges in IdeaSource
  • Idea sharing: Propose ideas for the challenge you feel closer to.Screenshot of the creation of ideas in IdeaSource
  • Reacting to ideas: Navigate and react to ideas with ease. Screenshot of the navigation of ideas in IdeaSource
  • Everything organized in your own profile: Check out the ideas you have created or liked, and the challenges you shared within your personal account.

Where can I get it?

The app can be downloaded from the App Store and Google Play Store, as well as it can be tried from the web browser. Regarding the code base, IdeaSource is meant to be an open source project for open innovation in organizations, so that it can be forked and modified, and for custom versions to deployed. In the following weeks we’ll be sharing the github code repo. In the meantime, we would love you to try it and provide feedback!!

Apr 08
3

ARKit 2: The Good, the Bad and the Ugly

By | Augmented Reality

Image of an iPhone showing a virtual pokemonARKit 2 is the latest framework from Apple that allows developers to work with Augmented Reality in iOS. When it launched in 2018, ARKit 2 brought new features and improvements in comparison with the previous version, but yet it has a long way to go. Our dev team spent some days researching and playing with it in order to find out how far it could go, and here are the conclusions.

Which things does this framework do well? And which ones not so much? Let’s take a look at the Good, the Bad and the Ugly of ARKit 2.

 

The Good about ARKit 2

 

Plane detection technology

ARKit 2 from stock includes many different tools in order to create a rich AR experience. One of them is the ability to detect horizontal and vertical planes by using the device’s camera and tracking what is known as features points. When the framework algorithm recognizes these points aligned in a real-world flat surface, it returns this information as an ARPlaneAnchor object, which contains data of that plane and represents the link between the real one and its virtual counterpart to be tracked throughout the AR session.

This information includes the general geometry of the detected plane, along with its center and dimensions, these being updated as new information from reality is discovered by the device. On supported iOS devices (with A12 or later GPU), ARKit attempts to classify detected planes into the categories of walls, floor, seats, ceiling, tables or none if no of the listed corresponds.  This additional data that the algorithm reports to the application might be of interest, for instance, to spawn certain objects on floors and not over every detected horizontal plane, allowing a more engaging AR experience.

Spawning virtual objects with their relative movement

As part of an enhanced augmented reality experience, virtual objects should behave as closely as possible to real ones. In particular, relative movement and parallax effects allow this greatly, simulating that objects are placed in the world itself and not just on the device’s screen with no relation to its surrounding. ARKit 2 is very successful at this task, tracking the user’s movements throughout the AR session and adjusting the virtual objects positions in consequence, relative to a set of initial coordinates.

Screen recording from an iPhone showing a spawned virtual Squirtle

This requires almost no effort at all by the developer’s hand, with the template being created as soon as you start a new project, showing an object (a spaceship asset) placed in the scene with the capability of letting the user move around it and see it from all sides. Therefore, this is a very strong feature of ARKit 2 especially considering how easy it is to work with.

Recognizing exact images

With our research of the ARKit 2 framework, we looked for solutions for image recognition in order to add virtual objects, like texts or even videos to identified images (see the example below). A use case for this feature could be showing information about a company when its logo is recognized. ARKit has a built-in image recognition system to which you can feed digital images that will be then recognized in the real world. SceneKit then allows you to perform certain actions once the image has been recognized by the software. This was accomplished with ease because, at the moment of loading the images, Xcode possesses an interface that analyzes them and gives the developer advice in regards of the image’s quality to ensure a more precise recognition. For instance, characteristics like size, colors or even design are taken to consideration.

After running several tests and following Xcode’s feedback, we could experience that the system does a reliable comparison of each image and makes sure they match exactly. Having said this, pictures such as landscapes with a wide variety of colors and shapes are identified in a good way as they are processed by their color histogram. Problems appear with simpler ones, something that we’ll discuss later in this post.

Realistic physical characteristics of an object

One of the approaches that we wished to cover was the ability of virtual objects to interact with each other and to have behaviors based on real-world physics. ARKit 2 brings the feature of adding physical characteristics to objects, like defining its mass. For instance, setting mass in virtual objects with oval bases such as a boat makes that when you move the plane where it is sustained the object will start to stagger.

Furthermore, the framework is capable of applying physical forces to virtual objects generating, in consequence, a reaction from it like moving forward or even falling if it is pushed away from the base-plane. This kind of physic properties improves the realistic appearance of virtual objects and the AR immersion.

Screen recording of an iPhone showing a virtual Squirtle falling from a table

Integral and adaptive architecture

In terms of its architecture, it might be worth mentioning that ARKit is similar (regarding its functionality into classes division) to other technologies that involve 3D object interaction, such as those pertaining to game development as the Unity engine. The concepts and terminology the framework handles, like Nodes, Anchors, PhysicsBodies, Shapes, among others, are present in some way and can be mapped to other technologies the developer might be more familiarized with, which is convenient in terms of speeding the learning process, pretty steep either way.

Apart from that, through our experimentation process, we found that it offers some possibilities of being integrated with other frameworks or libraries. In particular, for instance, a UIView (from iOS’s UIKit) can be placed in an augmented reality context, as a result of detecting a plane or image enabling then for buttons to be set in the user’s view and allowing interaction in this context or playing videos on some surface. Besides, when testing image recognition, it is possible to integrate ML Core which is an interesting option when extending the capabilities of ARKit in this sense is needed.

 

The Bad about ARKit 2

 

Plane recognition is not always accurate

 Even though the process of recognizing horizontal and vertical planes works fine in most cases, there are some edge cases where it doesn’t. When this happens, the feature points (that allow to identify planes) detected by the camera in the real world are not always “visible” because of certain circumstances of the context, such as the lighting, colors, and shadows.  For instance, a white wall or an uniform-color table don’t generate the specific characteristics needed for the camera to represent the feature points and to be able to work with. Take a look at the image below:

Images of a white table used as an example to ARKit plane recognition

We executed several tests where the goal was to identify a completely white table, with nothing over it, and it was impossible to recognize (image 3). Then we put some stuff on it, like pencils or even a keyboard, and after the software identifies them, it quickly did the same with the table (image 1-2, being the plane represented in brown). Although this may be related to limitations in the camera aspect of the device, we still think that there’s a lot of room for improvement.

Images identification only works well within strict parameters

Previously, we talked about the features that come with image recognition in ARKit 2, such as the ease of working with it or the feedback that the IDE provides in order to load identifiable images to the project. However, this is not a very flexible feature and minor changes in, say, the image’s color pattern make it difficult or impossible for the software to recognize it. Also, if the image is too simple, or it’s color range is too narrow, it becomes also very hard to identify and that is what Xcode is reporting in the image below:

Xcode's alert message about image recognition

As a solution to this problem, it would be interesting to include a trained system of machine learning in charge of recognition, letting ARKit only care about the AR part.

Bring your math skills

Creating a realistic AR experience, one that emulates real object behavior particularly if there is movement or physics involved, is not an easy task. Even though the ARKit engine takes on much of the hard work the level of math is probably beyond the typical dev comfort zone in order to implement a lifelike experience in the app.

For instance, one point that may cause some issues to implement correctly is, in the first place, converting from 2D screen/view coordinates to 3D ones in the virtual world (done through hit testing), especially considering each Node’s coordinates are given in relation to its parent’s coordinate system. Different data structures and types are used for all these coordinates, involving vector and matrix transformations to operate over them, which although efficient from a technical point of view, they are not easy to understand for someone new to this kind of domain as they involve mathematical concepts such as linear transformations and quaternions. Most of this is thankfully encapsulated in easy to call (but difficult to understand and master) function and methods.

Regarding the physics aspect of some possible AR project, several concepts are needed to configure bodies and joints correctly, as well as how they intertwine together such as mass, force, impulse, static and dynamic friction constants, etc. Even though it may be possible to set these through trial and error, some physics background is definitely helpful when working with ARKit in order to make interactions like collisions more realistic.

Finally, we found that at some points, due to jitter errors when receiving data from ARKit’s detection system, at the time of tracking the position of a certain image, for instance, or when plane anchor dimension’s change, it is at times useful to establish some error thresholds before updating or taking any action, and perhaps using linear interpolation to smooth out movements and make them more realistic. These are not that easy to implement in an effective way, making the framework as a whole quite hard to master.

ARKit hardware requirements

As it’s expected, this framework requires an iOS device with an A9 processor (iPhone 6s or later), and minimum iOS 11, while some features are only available with an A12 processor onwards.

Apart from that, in terms of phone resource consumption when executing an AR application programmed with ARKit, we noticed a significant drop in battery percentage after some minutes of use. This could be an interesting point to keep in mind when developing mobile applications in certain contexts, as battery consumption is a crucial aspect.

iOS’s only

As with many things Apple ARKit is an iOS only framework not available for Android apps. If you are serious about cross platform and feature parity you will need to bring the same capabilities to iOS & Android. Instead of having your team learn two frameworks and two ways of doing things you may want to evaluate implementing a library that supports both platforms, such as ARCore.

 

The Ugly about ARKit 2

 

Cannot identify whether the camera’s vision is hindered when spawning virtual objects

As we have said earlier in this post, spawning virtual objects is pretty easy and works fine. Problems start to appear when the camera’s vision gets hindered and that virtual object remains visible. The system is not able to realize that it shouldn’t display it as can be seen in the example below.

Screen recording of an iPhone using AR technology to spawn a virtual Squirtle

Granted, this problem is hard, and a proper implementation would probably require 2 cameras to measure depth. The ability to easily blend spawned objects with existing ones, remains a challenge not solved out of the box by the current ARKit version. Which totally breaks the illusion of the virtual object actually being there.

Poor object recognition (compile time & unreliable)

As it was mentioned before, ARKit offers the possibility of recognizing specific known objects or images in the real world. This works as long as the objects to recognize meet certain quality standards (high color contrast, no repetitive patterns, etc.) AND objects were scanned properly (from different angles, adjusting the bounding box as corresponds, etc.). Still, there are some key limitations in this, according to our testing.

The first is that both image files (with the .png extension, for instance), and 3D object data to be detected by the application, with the .arobject file extension, have to be included in compilation time as objects in a resource group to be handled by the application. This implies that all objects and images to be identified have to be known beforehand when developing the application itself. According to our investigation, ARKit offers no chance of detecting or categorizing 3D objects (eg. recognizing a certain object as a cup of coffee) other than this route, with the exception of plane surfaces. Below is shown the process of loading the information about a real object in order to allow ARKit to recognize it later:

Screen recording of the process of loading information to ARKit about a real object

The second is that ARKit uses an algorithm that compares the images captured by the device’s camera with such files, and returns a result based on how similar what is seen is to them, without any sort of artificial intelligence or learning behind the process. This means that detection suffers greatly when conditions vary, such as lighting, sight angle changes, etc. This is something to look out for if the detection of objects is crucial for the application in changing contexts, and can be improved either by improving the information ARKit is provided with, or possibly by integrating it with machine learning models trained in recognizing the desired object.

 

Final thoughts

These were our findings about the ARKit 2 framework after spending some days playing with it. Even though it brings interesting capabilities to mobile app development, it still feels a technology that’s in an infant state. Leaving many responsibilities up to the developer of things one would expect were solved, or at least approximated better out of the box.

Are you thinking about implementing AR in your next mobile project? Let’s get in touch!

Mar 21
4

Top 10 Mobile App Development Trends for 2019

By | Mobile Development

Each year brings its share of trends regarding technologies that push the envelope of what can be achieved in mobile app development. If you want to keep up with the mobile apps industry is important to take a look about what to expect for this year, and this post will help you with it!

We consider these to be among the top trends shaping mobile app development in 2019.

 

1) Artificial Intelligence: Chatbots and Voice Assistants

The presence of AI in our mobile devices is something that has been around for some years now, bringing automated assistants that can perform tasks and interact with real people, like Siri from Apple.  These assistants are improving every year by learning user behaviors in order to provide a more accurate and personalized experience. For instance, Google recently launched an AI project called Google Duplex that allows users to ask their phone’s Google Assistant to make reservations by phone call, simulating a human voice and being capable of having a fluent conversation with the person on the other side of the line.

Illustrative image of Artificial Intelligence
The same happens with chatbots, which allow businesses to automate the most frequent interactions saving humans time, and reducing cost for the enterprise. It’s estimated that by 2020, 85% of customer interactions will be handled without a human according to Gartner. Even Facebook announced that last year more than 300,000 bots were used by enterprises over Messenger, and the API to enable chatbots in Whatsapp was introduced a few months ago.

Chatbot platforms make it easier to implement chatbot based solutions, and users are becoming every day more used to text & voice based interactions with apps and software in general. It is in a way a new frontier in user interfaces, and it will affect the development of custom mobile apps. Be it as a way to enable hands off interaction with an app, or as a front-end to interact with several back-end systems through a chat based interaction, or as way for apps to respond to commands that were triggered through a voice assistant, such as Siri shortcuts.  This is why mobile developers must keep a close watch on chatbot and voice assistants.

 

2) Augmented Reality

Augmented Reality (AR) technologies are not a new thing but they are becoming better and more pervasive every year, reaching unprecedented levels. Mobile games like Pokémon GO made this so popular that thousands of people were interacting with virtual objects and creatures that don’t exist but were displayed over the screen thanks to AR. However, applications for this technology can go even further in business critical fields such as: security, health, training, industrial maintenance and more.

Bulding model shown on an iPad by using AR technology

For instance, police officers in China are starting to wear smart glasses that allow them to identify suspects by face-recognition systems and even spot criminals in the crowd. Although they are still experimenting with it and working to exploit AR in the fight against crime. It is an interesting use case of applying AR in society’s matters and no doubt it will occupy an increasingly important place because, according to Statista, the number of AR glasses shipments worldwide will reach around 22.8 million units by 2022, and its market size is forecast to hit more than 20 billion U.S. dollars at the end of this year.

Apple has recently introduced the ARkit to foster augmented reality experiences in custom apps, based on the camera  capabilities of both the front and rear camera. Enabling iOS developers to add elements to the camera display in a way that makes those elements appear to inhabit the real world. This has huge applications in gaming, but also in enterprise information systems where real world objects such as business cards can be augmented with animated metadata on the fly.

 

3) The Internet of Things

Nowadays, almost every gadget or household appliance that you can think of is being built with Internet access. And not only for checking Red Sox’s last game score in the microwave screen while you heat the waffles, but for connecting with the rest of the house and electronic devices in order to become a unified system that can be managed by your smartphone. Jokes aside, the called “smart homes” are starting to grow in number and becoming a game-changing topic for mobile development.

"Smart Home" representation with the use of an iPad

This is because there is no better way to control your house behavior than by using your phone. Imagine reaching (or leaving) your home and being able to unlock doors, open windows and turn on lights with just one touch. Those things are possible now and keep being improved to allow domestic appliances perform more complex tasks, managing them with mobile apps such as Apple’s “Home”. In fact, according to IDC, investment in IoT is expected to surpass the $1 trillion mark by 2022, and the development of mobile apps for that purpose goes hand in hand.

In this sense Apple has introduced the HomeKit library in the iOS SDK bridging the gap between mobile developers and the sea of iOT protocols, and smart devices and gadgets out there.
 

4) Mobile Wallets

There was a time when we had to pay for everything in cash. Then credit and debit cards appeared so we didn’t need to carry a lot of money bills in our wallets. But there is a common denominator here, the wallet. That thing that we usually have in our pockets didn’t change. Until now.  The wallet of the future will be your own mobile phone, and that future is starting to become present.

An android phone being used in a payment terminal

Mobile payment systems such as Apple Pay are growing in usage and popularity. According to eMarketer, by the end of this year about 36% of the worldwide mobile users will use a mobile payment app. Maybe one of the most advanced markets in this sense is China where not only upper and middles classes are using mobile wallets for their daily purchases, but word is that even some beggars use the famous WeChat app to ask for money.

In this context mobile developers must be ready to add mobile payment capabilities to their toolbox. There’s Apple Pay, but that’s just one in a growing market of mobile ready payment gateways.  Also as in every other topic when something becomes more and more popular, related features and capabilities increase in importance as well. For example, elements like biometric, and multi factor user authentication in apps should be something to consider for mobile app developers.

 

5) Biometric Authentication in Mobile Devices

Using your  body to prove that you are actually yourself is a common feature of new mobile devices. In fact, several researches prove that more than 60% of smartphone shipments come with fingerprint scanning capabilities, and that number keeps growing. This kind of authentication technology is being improved every year, developing a faster and precise scanner that is now even built below the screen.

Fingerprint scanner

Biometric authentication in mobile devices is now a mainstream. Not only used to unlock your phone but also to log into your bank account or any other sensitive digital space where you expect to have your data protected. Obviously, like any other security measure in the mobile world, it has its flaws, but biometrics combined with classic authentication methods represent state of the art authentication security. Mobile developers must be ready to include technologies such as: FaceID, or TouchID capabilities to their  mobile apps.

 

6) Accelerated Mobile Pages (AMP) and Progressive Web Applications (PWA)

Making a fast and user-friendly website that performs well in any device (especially in mobiles) is one of the main goals for developers. People leave sites that take too much time to load, and the same happens with applications. AMP and PWA bring new ways to tackle and old problem.

Programming code shown on a MacBook

AMP (Accelerated Mobile Pages) is an open-source library that provides a way to boost the performance of a web page and to improve its load time. It was introduced some years ago and gained popularity among web development because of its effectiveness. On the other hand, Progressive Web Applications are web apps that behave just like a mobile native app in order to create a reliable, fast and engaging experience for users. The advantage of this technology is a cheaper development cost comparing to developing a traditional native application, but with closer results.

AMP and PWA can be integrated together to take the best of both worlds, and that is why mobile devs are working with them having in mind that most of the web browsing occurs in a mobile device. Not only they enhance user engagement in your website, but also improve your visibility over search engines getting more traffic as a result. Because of this, both technologies form an important trend in mobile app development for 2019 that you shouldn’t miss.

 

7) 5G Technology


The next generation of cellular networks is hitting the door, and it is expected to reach the market by 2020. Experimental tests and researches hold that 5G is faster, cheaper and more stable than 4G, consuming even less energy. This is why there are already mobile phones being developed with this brand-new technology in order to be prepared for its arrival, such as the Samsung Galaxy or the Moto Z3.

Illustrative image of the 5G network

Even though everything is still too recent, mobile apps and the device as well needs to fit the 5G requirements, because we are talking about the network of the future. According to Statista, the number of 5G connections is estimated to reach between 20 million and 100 million by 2021. And a mobile device can lose quickly its user-engagement if it is not suited for that new technology meaning it won’t have a connection speed as fast as the competence.

So, what does 5G mean for mobile developers? Well, among other things: improved content, and specially video based experiences, including 4k video streaming, but also bandwidth to better handle heavy VR & AR 3D objects. Not to mention app bundle size becoming less of an issue, and more users willing to work on the go as the gap between mobile and wifi connectivity becomes less noticeable.

 

8) Foldable Displays

The breach between phones and tablets is merging into a new category; the foldable phones. These ones are starting to become  known and the first models are expected to appear this year, like Samsung’s Galaxy Fold and Huawei Mate X. DSCC (Display Supply Chain Consultants) forecast that in 2019, around 1.65 million foldable OLED panels will be shipped worldwide, reaching 60 million units by 2022 what shows a clearly fast grow.

A photo of the Huawei Mate X

For the time being, this is only a thing in the Android side of things, but it will have implications when developing user interfaces. Mobile apps will probably need to be optimized for that instant change of screen size between the folded version and the unfolded one. It will introduce yet more complexity and dispersion to the already crowded space of different screen sizes in the Android world, maybe opening more opportunities for players that automate the generation of user interfaces in apps.

 

9) Beacons & Nearby Networking

Even though the Beacons technology has been established for some time now, it didn’t reach its full potential yet. Beacons are devices that emit BLE (Bluetooth Low Energy) radio signals that smartphones can see and in most cases respond to. For example, stores can take advantage of this by sending day offers or information to customers that walk into the store, through their own app.

Illustrative image of a smartphone receiving Bluetooth waves.

That’s not the only thing BLE could work for. In fact, we have created ourselves a nearby messaging app based on BLE, and it’s also heavily used in mobile payments scenarios. It remains to be seen whether BLE and beacons in general will take off in 2019.
 

10) Mature hybrid mobile app development frameworks

The iTunes app store has turned 11 this year. After an initial cambrian period of exploration the market has settled several years ago on two main mobile platforms, iOS and Android. These platforms have matured and in turn this has enabled a number of frameworks such as: React-Native, Flutter, Xamarin, Kotlin-Native to evolve over the years. The problem of managing and keeping up with multiple code bases for one mobile app is a tough problem. Any one who has struggled to achieve and sustain feature parity providing a common user experience among platforms knows this.

Image of an Android phone showing its apps.

This is not a breaking trend but more like a creep, or a steady rise of what represents a viable alternative to native development. The space of user experiences only achievable through native development is slowly but steadily becoming thinner over the years.  There will probably always be experiences that only native mobile development will be able to achieve, but experience in some hybrid development framework should be part of the toolbox of any mobile app developer this year. Particularly for developing MVPs, or any early stage product release where product market fit is not yet assured. In these contexts hybrid development frameworks provide an increasingly solid path to save money, time, and headaches.
 

Final thoughts

The list of trends mentioned in this post is based on research and the opinion of our mobile team, but it’s not by any means extensive. Other trends did not make the cut, such as instant apps, and blockchain in mobile, but what do you think? What are your thoughts about this year trends? Leave a comment below and tell us!

Mar 15
2

Generating REST web services from PostgreSQL tables

By | Uncategorized

We’ve just released another open source project in github, a humble utility we’ve been using to generate and maintain data intensive rest endpoints that have a CRUD behavior on database tables.

It’s implemented in Java, and it generates:

  • Java Beans for tables.
  • JDBC based Data Access Objects to retrieve and execute commands on the PostgreSQL database.
  • Jersey 2.0 endpoint classes for each table.

The goal was to automate the creation of straight forward endpoints, and to separate data structures, and queries affected by changes in the database in base classes meant to be inherited, and updated regularly as the database evolves. So that, every time you make a change in the database, you just run the generation process and it updates all your related code.

All the generation is driven by templates files, with some ugly tokens for dynamically replacing content and at times for required pre processing of conditional logic.

The generation is based on metadata queried from the PostgreSQL database, but it could be extended to support other relational sources.

It generates Java code, but we are thinking about moving to node.js REST endpoint generation, in the near future. Suggestions and contributions in this sense are welcomed.

Check it out, and please do provide feedback!

Mar 06
1

A slack alternative that runs on a Raspberry Pi

By | Uncategorized

We’ve made available a docker image of our enterprise chat, task manager (and slack alternative) Kezmo. We used a Raspberry Pi 3 as our infrastructure benchmark for a fresh installation ready to support a small team and it works 🙂

So, what’s Kezmo?

Kezmo is an enterprise chat and team collaboration platform that can be customized and integrated to any system in your organization. It’s based on a small set of core concepts, which are: work spaces, conversations, containers, and container items. A workspace is a pocket of context to collaborate around any given subject. It can be a team, a project, an short term initiative, a client, you name it. Members can be invited to workspaces by it’s creator, where they can exchange conversations, documents, notes, tasks which are stored in workspace containers. The underlying design goal was to provide an intelligent way to store and categorize shared content by type,  attempting to overcome chat archeology whenever you need to go back and recall what was the agreement, link, or file shared at any time.

You can try the vanilla version from the cloud, or you can run it on premises. We implement white label versions of Kezmo, with custom content types, forms, and automations to integrate it into domain specific systems, in health, finance, and construction. If this could interest you let us know.

Where can I get it?

You can try the vanilla version from the cloud by signing up here, or your can try it in your own premises with the docker image we’ve recently released. We would really appreciate any feedback on the setup process.

Jun 26
5

How to build your customer support chatbot with DialogFlow

By | Artificial Intelligence, Chatbot, Machine Learning

We have recently released our support chatbot in Kezmo (our own @kezmobot), implemented with DialogFlow (former API.AI), and this blog post details some of the lessons we learned building it.

Kezmo is a task management & messaging platform for organizations that require an agile internal and external communication.  The product includes a support space where users can chat with our team. Up until recently support was handled by a subset of the whole dev team. While this has been great as a way to get insight into our product’s usage it’s also true that pareto holds and 80% of the users tend to ask about the same things, hitting 20% of our knowledge base. Also, answering customer questions during weekend nights is not fun after a while.

So we set out to build our own customer support chatbot.

Why API.AI?

If you are about to build a chatbot you’ll learn there are number of alternatives out there provided by the major players in technology, and then some more. Among them: API.AI from Google (recently acquired), Microsoft Bot Framework, IBM Watson.

API.AI was recommended to us by our good friends at the Tryolabs team. When evaluating alternatives such as the Microsoft Bot Framework we realized that API.AI requires less code plumbing, and overall proposes a more simple approach (higher limits on number of intents, no need for handlers on intent callbacks, no multiple cognitive services and endpoints). Plus API.AI is free, right? After exchanging emails with Google’s API.AI sales team, I’ve learned that:

Our platform is free to use as provided but we do restrict requests per minute to 180 on average. Google Cloud will add paid tiers later in 2017 with new features that align with enterprise requirements and allow for increased request rates.

They said the free version isn’t going away. We’ll see about that. So, for now free means you can make up to 180 queries to API.AI per minute, so you get about two per second. For the time being this works for us.

Getting started with API.AI

In API.AI chatbots are defined as agents. In API.AI you define an agent based on intents, which are the conversations interests it can handle. For each intent you define a number of phrases, texts, used to train the model. So that whenever a similar phrase is found that intent is matched. In our experience, in API.AI you should think about loading around 20 sample sentences for each intent for the model to behave reasonably good.

You connect to your agent using any of the supported out-of-the-box channels, such as Facebook Messenger, or Skype, or you can code your own client using any of the supported libraries to connect with API.AI. Since we wanted our bot to run on Kezmo, our own messaging platform, we had to program the integration.

When you create an agent you can base off existing sample data which preloads the agents with typical intents, or you can import prebuilt agents, which also preload intents into the new agent.

We wanted our bot to handle basic small talk, so we imported the small talk pre built agent, which loaded 84 intents. The imported intents do not include preloaded answers. So expect to spend some hours loading content in order to enable casual conversation skills.

Then we moved on to load our FAQ knowledge base as intent responses. In order to do this we combed all our support conversations for the past six months to identify intent patterns, and mine typical phrases used to express intents. This took days. We ended up with more than +500 sentences expressing intents.

Output formatting of the chatbot responses in API.AI

For a first version we loaded the answers directly to API.AI, I know we could have implemented an action, and a fulfillment to retrieve them from some endpoint but not for version one (make it work, make it right, make it scale).

The first issue we faced was supporting line breaks in the response texts loaded in API.AI. For some reason this gets lost in the client call, so we had to include markup (\n) and a basic post processing code for the API.AI calls to replace \n occurrences for actual line breaks in our messaging platform.

Implementing the bot human handoff with API.AI

Kezmobot is a customer support agent meant to act as first level support, so we need it to be able to escalate questions in a number of scenarios, including when the user explicitly requests to talk to a real person. Our support channel is a group chat, which means it’s key that the bot remains quiet once the human agent takes over the conversation, otherwise you get the nightmare scenario where the chatbot answers both the client and the human support agent.

Transferring chat to a human agent is not something  API.AI supports out of the box. You need to add some client code in order to implement this sort of behavior.

In our case we relied on a hand off context, that whenever returned as output of an API.AI call the client code would simply sleep the bot for 10 minutes, for the current client conversation (it quickly became clear it needed to be more like 12 hours).

Since we wanted to achieve a natural looking transition we created an intent that would match phrases such as: @kezmobot I’ll take over from here. This intent outputs the QUIET-MODE context notifying the client code to stop passing calls for that conversation, and that’s how kezmobot goes silent.

Handling sentences with multiple intents with API.AI

After combing our support log we realized humans have a tendency to pack multiple intents in one sentence. Interactions like the following are common:

Hello, how are you doing? I wanted to ask about X feature, how does it work? By the way I love your product, thanks!

Here we have at least, 3 intents, like:

  • Opening Greeting
  • Feature question
  • Closing thank you

These type of sentences can lead to a mismatch of intents. The model may match any of the three intents, leaving two unanswered. What we did was implement a very basic pre processing of the text we send API.AI to recognize stop tokens, used to split the sentence in multiple calls. For instance, we would split the sentence on occurrences of substrings such as:  . ! ? which would output something like the following:

  • Hi
  • You can use feature X in this or that way.
  • Glad you like our product! (would be great if you could post a good review on the store!)

That’s half of the solution. The other half is adding in our welcome intent wording to let the user know our bot works best with short simple phrases 😉

Measuring your chatbot accuracy

We loaded a CSV file with two columns. On the left column a sample text from our support logs. On the right column the expected intent for each phrase. Then we ran a process that would call API.AI using each line in the CSV file, and check the output against the expected intent. The difference between expected, and actual was used to calculate the % of accuracy.

One caveat is that you have to sleep your calls to API.AI to avoid going beyond the allowed throughput for the free service.

Between executions we tuned the ML Settings for the agent. Basically we played with different numbers for the ML Classification Threshold. We settled (for now) on a 0.5 value, which worked best for us to avoid false positives.

Implementing a chatbot that can handle multiple languages with API.AI

A big part of Kezmo’s user base is from spanish speaking countries, so it was key for us to support multiple languages.

They way to implement this is using multiple agents, one trained for each language.

Currently we have released the bot for spanish, and we are working on tunning the english version of it.

The good thing is that with our approach the client code does not care about the spoken language, or even the intents at play. We rely only on existing contexts, that must be shared among languages. What sets them up is transparent for the client code.

Chatbot analytics on API.AI

For the time being API.AI analytics are pretty basic. It’s rumored that they will open Chatbase to the masses some time. In the meantime log your own stats in your database.

Closing Thoughts

API.AI is adding features on a monthly basis. Business model is still not rock solid. But it’s simple, and gets the job done. It provides a good balance between expressive power and simplicity of the whole developer experience.

Real world usage of our chatbot is not like our human interactions, and now we are working on a second round of tuning after real interactions people are having with our bot.

If you want us to check your support process to see if we can implement a chatbot for it drop us a line, we’d be happy to help you!

Salvar

Salvar

Salvar

Salvar

May 30
4

How can you make UITableView / UICollectionView with dynamic headers

By | iOS, Mobile Development

Scroll based interfaces are everywhere. It has been the de facto overflow solution since we saw the first modern user interface, and of course it is widely used everyday on our mobile devices today.

But even though scrolls are very common, we have all seen apps spicing the scrolling experience up in different ways, and as mobile developers we want the best experience to delight our users too.

Having this in mind, I decided to create and share a simple UI component that hopefully will help you create better experiences as well.

Introducing the OLSDynamicHeaderViewController

OLSDynamicHeaderViewController is a simple iOS UI component that allows the most common scroll based UIs to have a header that can be easily animated along with user scrolling, as simple as that! So…

How do I get it?

The code is published on GitHub as you expect, with a sample project showing the basic capabilities. You can either use it through CocoaPods or you can also copy the files directly to your project, old school style.

How do I use it?

First, make your UIViewController to inherit either one of the helpers OLSDynamicHeaderTableViewController or OLSDynamicHeaderCollectionViewController.

 

Now, you need to provide a header view to the component, and for that, you need to implement the headerViewInstance() method and provide an instance of a OLSDynamicHeaderView subclass. This view must conform to OLSDynamicHeaderViewProtocol that consists in the following methods:

 

  • maxHeight() returns the maximum height allowed for this view.
  • minHeight() returns the minimum height allowed for this view.
  • resize(withProgress:) will be called when the user scrolls and your header needs to animate. `progress` goes from 0 to 1, 0 min height reached, 1 max height reached. This is probably the method you will be working the most to pull out a nice animation 😀
  • overflow(withPoints:) will be called when the user scrolls further the max height. `points` goes from 0 to N.

Cool! … anything else?

  • Note you can also inherit from OLSDynamicHeaderViewController directly if you do not want to use a UITableView or UICollectionView. The code of the corresponding helpers are there for you as reference of a base implementation 👍🏼
  • The sample code has a SampleHeaderView class that you can use (of course), but also have some basic math you may find handy for your header ideas!

Happy coding, cheers 🍻

Salvar

Nov 02
1

Will chatbots really kill the app store?

By | enterprise, Enterprise Collaboration
bot bender smokes stooggies

chatbot hype?

I recently came across the following article: Chatbots will soon kill the app store. In a nutshell the author states:

Messenger apps are huge already

Granted, we also believe this, that’s why we are investing in building Kezmo, a customizable messaging platform that can be setup on premises, so Enterprises can get the best of both worlds.

Chatbots are going to be huge (yyuuuuge)

We believe that chatbots will play a major role in the future of human computer interactions, and yes we think they are going to act as the trojan horse for the arrival of artificial intelligence. Still, the hype around the subject does not quite match the fundamentals. I don’t on a daily basis use or interact with any chatbot. The people I know don’t. Every chatbot interaction I’ve had has been in the context of toying with a new technology. I have not adopted it in any scenario yet, and I don’t know anyone who has. So, while I’m bullish on bots, and Kezmo will serve as the foundation for the deployments of bots either the technology is not quite there yet, or otherwise

The app store user experience is a mess

Indeed, to our dismay the app store has proven to be zero traction marketing channel. The fact that the majority of mobile apps are zombies is something we’ve experienced first hand.

Today 83% of the apps in the App Store are zombies

It doesn’t seem that far ago when free apps were able to generate hundreds of downloads per day just by the mere fact that they were free, and the app store was new. This dynamic is dead. If you are publishing anything to the app store, make sure to generate traction for it independently, relying only on the app store for marketing an app is not an option anymore, no matter how much app store search optimization time you put in.

Closing thought: are we comparing apples to oranges?

We can all agree the app store user experience is a mess, and both Apple and Google need to make changes asap to inject some life back in the app ecosystem. On the other hand chatbots seem like a promising investment despite imho the lack of true traction yet. To what extent are these two phenomenons related? Are we comparing apple to oranges?