The Smarter Camera

Mobile App Design


Smartphone cameras keep getting better with every version. So it shouldn't surprise you that while there are numerous apps that process photos and videos to make your media content look amazing, there are many others that use your smartphone camera for different purposes.

Being a writer, I would say that one of my favorite uses of phone cameras is converting images of typed up, printed or handwritten text, to digitized text. Your digitized text can now be stored electronically, edited, searched, and further processed. It can also help the visually impaired by using further text-to-speech processing to render an audio of the image's text. There are many iOS and Android apps out there that can perform this image-to-text conversion, some which are geared toward a specific user need, such as digitizing business cards. These apps use Optical Character Recognition (OCR) technology, a form of AI, which enables the text on the pre-processed image to be matched with letters and numbers, thereby converting it to digital machine-readable text.

This same technology can also be used in addition with further processing for varied applications, such as translating text on an image (for when you're in another country and all the transit signs are in the local language). Examples are Microsoft Translator and Google Translator, which use OCR in conjunction with text translation and machine learning.

Another interesting way in which the phone camera can be used to translate and learn languages is by using image recognition, (which is also a part of AI: read more here). The Snap and Translate app, for example, works like this: the user would take an image of the object that she wants to name in a different language; the app uses image recognition to identify the object, and then names it in the language of choosing.

Image recognition has so many ever-increasing applications, from self-driving cars to fighting crime. In the realm of mobile apps though, this technology is being heavily leveraged in retail in the following ways:

  • The App CamFind uses image recognition to identify an object and search for relevant content. I used the app to "identify" a coffee mug shaped like a snowman, and the app directed me to similar images as well as shopping links for similar "snowman mugs".

  • The scan-to-shop experience relies on the smartphone camera to scan barcodes and direct the user to the relevant product. The H&M mobile app, for example, lets the user scan a barcode in-store to find additional colors or sizes on their website. Price-checker apps like ShopSavvy use the camera to scan barcodes and compare prices from different retailers. Coupon apps like SnipSnap allow the user to take a picture of a printed coupon and store it digitally, and even reminds them of the coupon when they enter the relevant store.

  • Apps like MTailor also rely on the smartphone camera to take a measurement video of the user and ensure a perfect clothing fit. "Your phone's camera is now 20% more accurate than a professional tailor," they claim.

  • On the seller side, marketplace apps like Letgo use image recognition to automatically create a listing from a picture of the object to be sold.

  • And of course, the most important piece in the retail experience: using your phone camera to store your credit-card information, and using the camera to deposit that rebate check!

Augmented reality is another interesting use of the smartphone camera with leverages AI more heavily. We will explore this topic further in a separate article.

Let's not forget about video. It isn't just for Snapchat stories and Youtube,  video calling and conferencing. Video is now a powerful medium for marketing.

NoMoBo's Beehive app, for example, uses video to create a better, seamless and personal hiring experience with short videos between hiring managers and candidates, eliminating the time and cost of initial interviewing.

Beehive Talent Promo

How will your smartphone app use the camera? Contact us today and let us work with you to design, create and support your app!

Nidhi Jaisoor

Nidhi Jaisoor is a freelance technical writer based in San Jose, CA. She has a background in information technology and is interested in technology trends, user interface design, and the conscientious use of technology in making our lives better. When she isn't writing about technology, she loves writing short-story literary fiction. Nidhi is a graduate of Texas A&M.