CS491 - Virtual and Augmented Reality

Augmented Reality in Google Translate

Google translate is an application by google that helps translate texts to and from various languages of the world. In recent years, this app has developed immensely powerful features. Google translate has an intelligent implementation of Augmented Reality by which it has acquired the capability to translate any text found in any image in real-time while we see the text via camera display.

The following images show Google Translate at work. The application has a camera interface and has an instant translate option. It also has the ability to capture the images and analyze the text content and then translate them. Inspite of these innovative features, the main translate feature works with a less success rate for sentences. The first image and second image have different versions of translations for the same content.

Image 1Image 2

This capability can be added to AR applications too. In devices such as Google glass and Microsoft Hololens, this would open the window to a wide range of possibilities. Wearing any of these devices along with the power of Google Translate, a person’s abilities will never be limited by language barriers. People can move around the globe naturally without having to worry about languages. Real-time translations in Google translate would make the user read every content written in real world in his own language. So he never has to deal with the problem of translating text from other languages ever again.

This would become a reality if google translate has a hundred percent success rate. But sadly, it is far from becoming a reality. Natural Language Processing Capabilities are still at a level where real-time instant translations are not yet possible. Also, this app does not have the ability to identify the language based on the image. This is a very important capability needed for real-time translations. The user would not be knowing the language and he probably would not be able to identify it based on scripts.

This real-time translation is just one application of this. It might also be used for defense purposes, where even the soldiers at the frontline may be able to decipher information when they are stuck in an enemy territory. Though its not as useful as espionage, it will be useful for stranded soldiers.