Skip to main content

FlatBuffers instead of JSON (or Protocol Buffers)

FlatBuffers is an efficient cross platform serialization library for C++, Java, C#, Go, Python and JavaScript (C, PHP & Ruby in progress). It was originally created at Google for game development and other performance-critical applications.

Why use FlatBuffers?

  • Access to serialized data without parsing/unpacking - What sets FlatBuffers apart is that it represents hierarchical data in a flat binary buffer in such a way that it can still be accessed directly without parsing/unpacking, while also still supporting data structure evolution (forwards/backwards compatibility).
  • Memory efficiency and speed - The only memory needed to access your data is that of the buffer. It requires 0 additional allocations (in C++, other languages may vary). FlatBuffers is also very suitable for use with mmap (or streaming), requiring only part of the buffer to be in memory. Access is close to the speed of raw struct access with only one extra indirection (a kind of vtable) to allow for format evolution and optional fields. It is aimed at projects where spending time and space (many memory allocations) to be able to access or construct serialized data is undesirable, such as in games or any other performance sensitive applications. See the benchmarks for details.
  • Flexible - Optional fields means not only do you get great forwards and backwards compatibility (increasingly important for long-lived games: don't have to update all data with each new version!). It also means you have a lot of choice in what data you write and what data you don't, and how you design data structures.
  • Tiny code footprint - Small amounts of generated code, and just a single small header as the minimum dependency, which is very easy to integrate. Again, see the benchmark section for details.
  • Strongly typed - Errors happen at compile time rather than manually having to write repetitive and error prone run-time checks. Useful code can be generated for you.
  • Convenient to use - Generated C++ code allows for terse access & construction code. Then there's optional functionality for parsing schemas and JSON-like text representations at runtime efficiently if needed (faster and more memory efficient than other JSON parsers). Java and Go code supports object-reuse. C# has efficient struct based accessors.
  • Cross platform code with no dependencies - C++ code will work with any recent gcc/clang and VS2010. Comes with build files for the tests & samples (Android .mk files, and cmake for all other platforms).
Why not use Protocol Buffers, or .. ?

Protocol Buffers is indeed relatively similar to FlatBuffers, with the primary difference being that FlatBuffers does not need a parsing/ unpacking step to a secondary representation before you can access data, often coupled with per-object memory allocation. The code is an order of magnitude bigger, too. Protocol Buffers has neither optional text import/export nor schema language features like unions.


FlatBuffers on GitHub

Improving Facebook's performance on Android with FlatBuffers
FlatBuffers is a data format that removes the need for data transformation between storage and the UI. In adopting it, we have also driven additional architectural improvements in our app like Flat Models. The mutation extensions that we built on top of FlatBuffers allow us to track server data, mutations, and local state all in a single structure, which has allowed us to simplify our data model and expose a unified API to our UI components.

In last six months, we have transitioned most of Facebook on Android to use FlatBuffers as the storage format. Some performance improvement numbers include:


  • Story load time from disk cache is reduced from 35 ms to 4 ms per story.
  • Transient memory allocations are reduced by 75 percent.
  • Cold start time is improved by 10-15 percent.
  • We have reduced storage size by 15 percent.

FlatBuffersSwift on GitHub
Infrastructure for FlatBuffers, contains a reader and a builder

Comments

Most Favorite Posts

Dark Theme (Dark Mode) in Android WebViews, WKWebViews and CSS

So your apps just implemented a shiny new dark theme and it’s looking đź‘Ś There are lots of benefits to having a dark theme in your application, and having it consistent throughout your application allows for a great user experience. But what happens when the the user runs into a WebView in your app? Support: if (WebViewFeature.isFeatureSupported(WebViewFeature.FORCE_DARK)) { ... } Set: WebSettingsCompat.setForceDark(webView.settings, WebSettingsCompat.FORCE_DARK_ON) Current setting: val forceDarkMode = WebSettingsCompat.getForceDark(webView.settings) Joe Birch Assuming your question is asking how to change the colors of the HTML content you are displaying in a WKWebView based on whether light or dark mode is in effect, there is nothing you do in your app's code. All changes need to be in the CSS being used by your HTML content. CSS dark mode via :root variables, explicit colors and @media query: :root {     color-scheme: light dark;         --h1-color: #333;

Server-driven UI (SDUI): Meet Zalandos AppCraft and AirBnB Lona

A short WTF: Joe Birch:  SERVER DRIVEN UI, PART 1: THE CONCEPT Zalando seems to follow the SDUI principle as well - defining a common design language and construct the screens on the backend while displaying them natively on the clients. They even go one step further; they implemented a mighty toolset to enable non-technical stakeholders to define their own native app screens Compass: Web tooling to create screens and bind data Beetroot: Backend service that combines the screen layout definition with the data Lapis/Golem: iOS/Android UI render engines Crazy cool! Good job, guys (when you do an open-source release?) To even move faster a Flutter based UI render engine implementation was great! See also AirBnB Lona SDUI approach Building a Visual Language Why Dropbox sunsetted its universal C++ mobile project and AirBnB its React Native implementation

Judo App - Server Driven UI out of the box

Judo App Judo brings server-driven UI to your iOS and Android apps. Build user interfaces visually in a fraction of time and publish them instantly without submitting to the app store. Build Experiences - With No Code The Judo app for macOS, available through the App Store, is built for design professionals with common keyboard shortcuts and familiar concepts like canvas, layers and inspector panel. Workflow is streamlined with the ability to drag and drop media files directly into your experiences and manage your own Judo files in Finder. Manage Creative Execution A Judo experience is interactive and can include text, images, video and buttons. An experience may be part of a screen, a single screen, or more typically multiple linked screens. Judo supports screen transitions, carousels, horizontal scrolling and modals. Clients can add custom fonts and define global colors and these are updates applied universally. Effortlessly Deploy Judo Cloud syncs your experiences with your iOS and

Backend-driven native UIs

Backend-drive native UIs John Sundell  Slide Share Using Back-End Design to Create Customizable Front-End Mobile Experiences By controlling the front end of mobile apps from the back end we can build customized experiences at runtime, creating cleaner interfaces and reducing load times. Nithin Rao UX Magazine The Hub Framework Welcome to the Hub Framework - a toolkit for building native, component-driven UIs on iOS ( no Android support released yet ). It is designed to enable teams of any size to quickly build, tweak and ship new UI features, in either new or existing apps. It also makes it easy to build backend-driven UIs. The Hub Framework has two core concepts - Components & Content Operations. Spotify LeeGo: Build UI without UIView LeeGo is a lightweight Swift framework that helps you decouple & modularise your UI component into small pieces of LEGO style's bricks, to make UI development declarative, configurable and highly reusable. Wang Sheng Jia

Firebase Performance Monitoring

Introducing Firebase Performance Monitoring Monitoring performance from the end user's point of view - both app code performance and network responsiveness/reliability - is a challenge for building great mobile apps. You have to be able to understand conditions that need improvement, in order to avoid user churn and negative reviews. Conversely, we have seen that 60% of 5-star reviews on Google Play mention speed, design or usability. This is why we built Firebase Performance Monitoring for iOS and Android. It provides an SDK to capture and instrument performance metrics in production, and a console to gain insights about the data captured by the SDK. Firebase Google Blog Firebase Performance Monitoring Docs

Android Material Design Icon Generator Plugin

This plugin help you to set material design icons to your Android project. GitHub konifar

ImageOptim

ImageOptim optimizes images — so they take up less disk space and load faster — by finding best compression parameters and by removing unnecessary comments and color profiles. It handles PNG, JPEG and GIF animations. ImageOptim seamlessly integrates various optimisation tools: PNGOUT, AdvPNG, Pngcrush, extended OptiPNG, JpegOptim, jpegrescan, jpegtran, and Gifsicle. It solves too the Xcode bug: corrupt PNG file. ImageOptim