Bitsletter #7: Accelerating ML Training & Diving into Geometry-aware 3D GANs
๐ง ML Tip: Pay attention to data loading
Data loading is an important part of any machine learning project. Models need data, and data must be carefully prepared and fed to them. Training time is another important factor in machine learning: the quicker is your training loop, the most examples you can show to your model for the same time budget. Nowadays, access to accelerated hardware such as GPUs is ubiquitous, it's also mandatory to train big models on big datasets in a decent amount of time.
However, it's not as easy as attaching a GPU to your virtual machine. What's the point of using a GPU if your utilization rate is low? Indeed, GPUs are so fast at matrix multiplications that your data loading process must transfer data to them as efficiently as possible. Otherwise the GPU will spend most of its time waiting for data. If the data fits into your memory, you should not have much trouble. The CPU should be able to transfer the data quickly to the GPU (except if you do a lot of CPU bound preprocessing before sending the data).
However, in most cases, the data doesn't fit entirely in your memory, and your data loading becomes IO-bound: you spend time reading data from the disk every time you build a batch for the GPU. Most of the time, if you perform data loading naively, you won't optimize these IO calls, resulting in a poor utilization of your GPU. Techniques such as moving preprocessing inside the GPU, pre-loading batches of data in advance or using memory mapped files can greatly improve your performance. Fortunately there exists libraries to help you implementing these tricks in you project, such as mmap.ninja or FFCV.
๐ Web Tip: Web Tip: You should use TypeScript
Javascript took over web frontend development with libraries such jQuery, React, Vue or Svelte. Projects like React Native even empowered web developers to build mobile applications. Javascript also became widely used in backend applications with the Node runtime.
That's great right? One language to build full-stack applications from frontend to backend. However it has some caveats. Javascript is a dynamically typed language with no static type checking at dev/build time. It's awesome at the beginning since you write code super fast.
However you quickly miss some features of statically types languages: great autocompletion, tons of bug detected at dev/build time, code quality, improved developer experience โฆ
Here comes TypeScript, as JavaScript superset and strongly typed language. It compiles to JavaScript so your code can run on any JS runtime. It brings you tons of benefits: static type checking catch many bugs to prevent crashes at run time. Ever forgot to check for null values? Not anymore with TypeScript. It also integrates better with your code editor, since the type information allow better tooling, autocompletion and more.
It's not that hard to pick up, and you will gain much more confidence in your code. Stop sweating when your continuous delivery triggers the deployment of your new release ๐ .
๐ฉโ๐ฌ Research Paper: Efficient Geometry-aware 3d Generative Adversarial Network (EG3D)
NVlabs released publicly the official PyTorch implementation of EG3D (CVPR 2022 Paper). Efficient Geometry-aware 3d Generative Adversarial Network (EG3D) is a recent paper about using GANs to generate multiple consistent view of images and 3D shapes. They only use 2D pictures has an input, which makes the problem particularly challenging.
Existing 3d GANs models are computationally intensive or sacrifice consistency by making approximations to reduce the computation time: it alters the multi-view consistency, meaning that a same object generated at different angles will clearly exhibits geometry or texture flaws. A key process to achieve their impressive results was to decouple feature generation and neural rendering to leverage state-of-the-art 2D CNN such as StyleGAN2. Definitely worth reading. ๐ก It's insightful to have access to the code produced by top AI labs. Reading others' code, is the best way to learn new techniques, tricks, useful libraries, ...
๐ Tool: Plasmo, build browser extensions easily
It's like NextJS but for browser extensions.
Who doesn't have any browser extensions? You likely have at least an ad-blocker. Browser extensions add features to the browsers. They allow you to integrate your products directly inside the Chrome, Firefox, Safari, ... of your users. Plasmo is a framework to ease the development process of browser extensions.
- ๐๐ฝ First-class React + TypeScript Support
- ๐๐ฝ Declarative development with manifest.json auto-generation
- ๐๐ฝ Live-reloading
- ๐๐ฝ .env* files
- ๐๐ฝ Remote code bundling
- ๐๐ฝ Automated deployment
Do not hesitate anymore. If you application can benefit from a browser extension, use Plasmo and build it in a glimpse.
๐ฐ News
Grid.ai
Lightning AI is the startup behind the PyTorch Lightning framework which dramatically reduces the boilerplate of PyTorch projects.
The startup also provide a SaaS to easily scale your machine learning workloads in the cloud and leverage distributed training using many GPUs.
Push API: Web Push Notifications coming to iOS and Safari
Apple is working on the Web Push Notifications for iOS and Safari. As part of the iOS 16 release, Safari will support the Push API. This is a great new for web developers, since the Push API is becoming a standard for sending notifications to your users.