From where we started, Gojek has grown to be a community of more than one million drivers with 3 Million+ orders every day in almost no time. To keep supporting this growth, hundreds of microservices run and communicate across multiple data centers to serve the best experience to our customers.
In this post, we’ll talk about our approach of assembling Infrastructure As Code that simplifies the maintenance of an increasingly complex microservices architecture for our company.
Building infrastructure is, without a doubt, a complex problem evolving over time. …
At GO-JEK, we build products that help millions of Indonesians commute, shop, eat and pay, daily. The Data Engineering team is responsible to create a reliable data infrastructure across all of GO-JEK’s 18+ products.
We aim to provide disproportionately large advantages to GO-JEK over its competitors by making data available, accessible, reliable and actionable at scale.
The GO-DATA platform allows internal teams to build on our products to develop innovative solutions; creating new opportunities for our customers and expanding on what’s possible.
Some informal philosophies and approaches we follow during our time working on data infrastructure:
Data at GO-JEK doesn’t…
At GO-JEK, the Data Engineering team is responsible for crafting reliable data infrastructure across all of GO-JEK’s 18+ products. We aim to provide disproportionately large advantages to GO-JEK over competitors by making data available, accessible, reliable and actionable at scale.
Our GO-DATA platform allows teams to build on our products to develop innovative solutions, creating new opportunities for our customers and expanding what’s possible.
In the real world, when we buy a product, it comes with a quick guide or some other form of documents which help you get going with the product. …
Given that our internal services generate billions of events a day, our data infrastructure must be highly scalable, available, and flexible enough to keep up with rapid product iteration and exponential data growth. This results in spending a substantial portion of our time in infrastructure provisioning, load testing, and system recovery. This article describes how we envisioned Asgard, our toolbox for automating end-to-end data infrastructure challenges.
Every product needs a vision. This can come from many sources; most important being solving existing pain points and fulfilling a real, deeply-felt human need. …
It’s important for us to build a resilient and scalable data infrastructure when we have 18+ products. This means designing systems that can sustain operations in the face of failure.
Built on the principles of Chaos Engineering, and tailored to our specific use cases, Loki is our internal disaster simulation and load testing tool. It helps ensure our data infrastructure can tolerate random instance failures. Exposing engineers to failures frequently incentivises them to build these resilient services.
At GO-JEK, we build products that help millions of Indonesians commute, shop, eat and pay, daily. The ‘Data Engineering’ (DE) team is responsible to create a reliable data infrastructure across all of GO-JEK’s 18+ products. In the process of realising this vision, here are some learnings:
At GO-JEK, location is built into the fabric of all our products, powering dozens of products that are used by millions of customers and drivers, daily.
Here’s a tweet from one of our investors from when this post was written to help contextualise our scale.
As a result, we have billions of GPS points flowing through our data pipelines daily in real-time. This data drives decisions like driver allocation, surge pricing, driver incentives and many more.
At GO-JEK’s Data Engineering team we built Atlas in an attempt to make it easy for teams within GO-JEK to visually explore this…
In a hyper-growth organization like GO-JEK, technology plays a vital role. However, as an organization matures, the hard part is not scaling the technology, but paying heed to culture.
For us at GO-JEK, culture is a collective philosophy about how to build products that change lives. We all do our bit to make sure it‘s transparent and open to innovation.
A great way to understand an organization is to ask, “Why should someone work there?”
For me, it’s the people. And it’s not just me, most of us at GO-JEK will have a similar answer. …
Throughout my career, I’ve have changed job roles more frequently than I have changed buses.
I started out being a tutor, then sold health insurance, marketed embedded products, designed interfaces, developed embedded systems, wrote software, wrote stories as a data journalist, was an entrepreneur and much more.
As a software developer, specifically, I’ve been doing prototyping, design, frontend development, backend development and now exploring devops and data engineering. And that must be the reason that I’m frequently asked: How to do it all and how to be a full stack dev.
So, here are a few techniques that have been…
Just another day in the life of one.
August 2014, I was in the process of evaluating the tech stack for my next project. For the last few projects, I used MEAN stack, the most recent being Resumite, and I loved working on Express and Angular, but now it was time to move on, as the future of both seems very doubtful.
5:00 am, and I was exploring the newly discovered world of Isomorphic JS with React, flux, and whatnot. Being a JS lover, I am fascinated with the future of the web's isomorphic/universal era.
Web apps started with full…
Data Engineering, Data Visualization, Infrastructure Automation, Distributed systems