Today, technology is evolving faster than ever. And with the rapid pace, one needs to be on the top of technological advancement happening around the world. We, at Ninjacart, always believe that technology can change the world for a better tomorrow.
Table Of Content
Here, we discuss how we can utilize various OpenSource projects to create a low-cost & highly available system. The OpenSource Projects have been a backbone for millions of developers and hundreds of companies around the globe. We will pick some of the popular projects today to create a robust pipeline.
We would aim to build the stack by piecing together different technologies. Here, we are discussing the different possibilities to optimize the sequence of operations applied to complete the request paths. We will walk through routing the request, configuring the servers with monitoring, logging and alerting.
So, let’s get started.
EntryPoint (LoadBalancer/Gateway/ReverseProxy)
The first thing we build any application for is to receive traffic from users. It can be a mobile or a Web Application, but the backend would need a point of entry to exchange the messages.
As most businesses start by buying a domain, we did the same as well – pick a domain name and a domain hosting provider (GoDaddy, AWS, Google, etc). Once you get a domain, you can point it to one of the public-facing nodes which can expose PORT 80/443 to resolve the requests. You can also create/use a subdomain.
You should also block other unnecessary traffic. This step is critical, otherwise, you will be exposed to system vulnerabilities for attackers to enter the system. You may like using simpler tools like HaProxy or Nginx to resolve the requests. You should block unwanted traffic which can include HTTP-1.0, HTTP-1.1, SSL-V1/V2, etc. You can also configure DDOS, HSTS, HTTPs here.
. . .
Application Server
There are various options available to host the application. One can choose between cloud vendors to on-premise computing. You may like to stay cloud agnostic to avoid overdependence on vendors. Application containerization may help avoid such issues. For example, by using Docker one can run Orchestration on bare metal, achieve high service availability and scalability. You may decide to host your project images to a private server registry using open source edition or make it public using docker hub.
Be it running clusters using docker or independent servers on virtual machines, the aim is to run the business solution with high uptime and visibility. Uptime can be impacted by multiple factors like a number of bad deployments, memory leaks, managing concurrent connections, database queries, etc.
A typical modern application may look like a mesh. An application typically connects to primary datasource (Mysql/MongoDB/cassandra/postgres), caching systems like Redis, Queues like kafka/SQS, authentication module, configuration management tools, credential vault, etc.
Given the increasing complexity of modern applications, monitoring such applications becomes critical. One may again choose between licensed tools like Newrelic or choose free and open source applications like pinpoint.
. . .
System Monitoring and Alerting
A robust monitoring pipeline not only provides timely data but also ensures scalability of the systems is never compromised. One such pipeline can look like this —
Application registration happens automatically to the consul client running on application host server. Consul extracts essential scraping details like metrics path, application type, instance id, health checks, etc. This information is shared with the consul leader. Any client with required privileges can request and supply this data to prometheus which starts to scrape the information and store it in.
Prometheus being a time series database helps to track events in time easily. You may start an instance of grafana and connect prometheus as datasource. Grafana does support alerting out of the box. Hence you may create queries for dashboard and grafana can report the abnormality or any breach in thresholds.
With this, we would conclude that by bringing together all the tools mentioned above can be proven effective to build a scalable and highly available system with minimal maintenance and cost. We at Ninjacart have tried and tested multiple such stacks and run the above stack with confidence.
. . .
For more information, Feel free to connect back to us and we would be happy to help.
Written by
Sandeep Sogani
SDE-II (Platform & Infrastructure)
Tech Team – Ninjacart
5
No Comment! Be the first one.