Vuze Camera – Application for a Remote Control

Business Domain: Information Technology and Services

Project Type: Mobile App, IoT

Business overview

The client is a company that is an expert in 3D, computer vision and image processing with 70+ patents. The product is the first consumer 360 3D camera and the new dual-camera that gives anyone the power to create and share immersive experiences in 360° or VR180 (3D) Photo and Video.

We aimed to develop a mobile application for Android and iOS which allows users to operate the camera, unlock the full potential by enabling different shooting modes and powerful editor.

Challenge

  • Develop a 3D video editor with Little Planet effect and various photo filters;

  • Create Cardboard mode and VR mode;

  • Assure constant connection between the client device(as an access point with separate WIFI) and mobile application;

  • Provide the ability to edit video on the fly (when playing back) for high-resolution media;

  • Ensure stabilization and tracking of the camera;

  • Update the device firmware through the mobile app and ensure accurate operation with different firmware versions;

  • Implement video streaming to YouTube and Facebook with dynamic video quality changes and the ability to use the phone as an intermediate link.

Solution

Brick by brick, the team ensured a stable connection between the camera and mobile application. Using the socket connection and the Protobuf structured data transfer protocol allows channeling the commands from all mobile platforms to the device. 

Concerning the updates, the mobile application checks and downloads the new firmware version from the client’s servers and transmits it to the device starting the update process.

Main features of the Vuze Camera app:

  • Capture videos and photos in 3D 180 or 2D 360

  • Stream to Youtube and Facebook

  • Choose resolutions: Photo 18MP, Video 4K/30fps or 5.7K/30fps

  • Video stabilization and Google VR180 compatible format

  • Ability to watch media or live preview with VR glasses

  • Create little planet effects, make color adjustments, add audio

  • “Director’s Cut” mode enables to change the viewpoint, define how and where to start/end a video

  • Crop VR media to a selected ratio, and create classical 2D format videos

 

Outcome

The NIX team created an app with the hottest feature – the ability to make a 2D video from 3D / 360 ° footage. The application allows to view videos in Cardboard or Little Planet mode and make live-stream videos on Youtube or Facebook. Also, the powerful video editor enables many fun features such as: to add artistic filters or effects, make color adjustments, add audio.

It was a great adventure for us to develop such an innovative product and help the client to forge a path to success.

Tech: Swift, Objective-C, OpenGL, CoreML, Vision, AVFoundation, ProtoBuf, PromiseKit, C++, C, JNI, FFmpeg, EGL, OpenCv, boost, NDK, Kotlin, Java, GVR, FreeType, glm, libyuv, XMP, Crashlytics

Team: 17 (Project Manager, Business Analyst, 3 C++ Developers, 5 iOS Developers, 4 Android Developers, 3 QA Engineer)