Projects

Understanding its concepts and a little bit of CuteDSL.

Scaling Distributed GEMM on Cerebras Wafer-Scale Engine In Large Language Model (LLM), the fundamental operations of transformer architecture are attention and multi-layer perceptron computation, both of which are built on a massive amount of GEMM (General Matrix Multiply) and GEMV (General Matrix-Vector Multiplication). During inference, the decoding step (specifically GEMV) is memory-bandwidth bound due to the LLM autoregressive nature. (i.e. the GPU, such as NVIDIA’s accelerator, spends most of its time loading data into the compute unit for a relatively little computation, which computes one new token per step.) To address this issue, research has been attempting to transform this memory-bound workload to compute-bound by converting GEMV into GEMM operation through batching. However, memory movement is still relatively expensive. This observation of memory movement as the primary bottleneck, has encouraged companies like Cerebras to adopt a spatial dataflow architecture, in which compute units form a processing grid, enabling them to send data to one another at lower memory-transfer costs, thus addressing memory bound application from a hardware approach. ...

This study detailed the SGLang rabbit hole I went down.

A brief history of Computers The precursor of modern computers - the Analytical Engine - was proposed by Charles Babbage in the 1800s. Although it was never fully built, this is the first mechanical general-purpose computer, powered by steam, that featured a CPU-like processor, memory, and programmable input devices such as punched cards. This design laid the groundwork for concepts in general-purpose computing, including the separation of processing and storage. Around the same time, Ada Lovelace is the first to recognize that the Analytical Engine has applications beyond pure calculation, and is often recognized as the first computer programmer. During that time, a computer wouldn’t even need power to run, as it was just a huge mechanical engine with the core components of computers. ...

This blog includes my review on Inference Serving for Large Language Models as part of the requirements in the R244 Large-scale data processing and optimisation course.

This blog includes my review for TensorFlow as part of the requirements in the R244 Large-scale data processing and optimisation course.

This blog offers an engineer’s perspective on optimizing the performance of scientific applications on HPC heterogeneous systems, drawing from international HPC competition experience and an internship in NSCC.

MDP is a group-based design and development robot car project. This blog aims to provide a starting guide for incoming students who will be taking this course. The information is based on MDP AY24/25 Sem 2.