Server optimisation: indexes, caching and algorithms instead of buying hardware

Database optimisation with indexes, caching, better algorithms and higher server performance. Why adding capacity rarely solves the problem of a slow system.

Server optimisation: indexes, caching and algorithms instead of buying hardware
INGELLO / BUSINESS3044

This is a track from the corporate development block. It comes in as the product grows from MVP to MMP and MLP. The track costs $4700.

When a system slows down, the first instinct is to add resources. That works while the problem is a lack of capacity, and never works when the problem is a query that reads the whole table.

A slow system costs money every day. A manager waits for the deal list to load and gets through fewer leads. A customer leaves the catalogue page before the products appear. The owner's report takes so long to build that people stop opening it. Meanwhile the server bill grows, and users notice no difference. Server optimisation brings speed back without an endless hardware upgrade.

What is included

  • Database optimisation (indexes).
  • Caching.
  • Algorithm optimisation.
  • Server performance improvement.

Database optimisation

We find the queries that take longest or run most often and examine how the database executes them. We add missing indexes, rewrite queries that pull unnecessary data and remove repeated database calls inside loops. In the Ingello stack this is MySQL, and query optimisation is already part of the server-side business logic stage.

Caching

Results of heavy, rarely changing calculations are kept in a cache (Memcached in our stack) so they are not recalculated on every request. For each cache we decide in advance when it is cleared, otherwise users will see stale data.

Algorithms and the server

Sometimes it is not the database that is slow but the logic: a full scan where a lookup is needed, recalculating everything when one record changed. The server itself is tuned separately: background processes are moved out of user requests, and heavy tasks go into a queue.

The order of work matters more than the toolkit

Measure first, then change. Without measurement, optimisation turns into decorating the code: time goes by, and users notice nothing because the wrong place was fixed.

We measure what the business feels: the response time of key screens and operations, the slowest and most frequent database queries, and server load at peak hours. The same figures are taken again after the changes, so it is clear what the optimisation actually achieved.

Caching comes last, not first. It hides the symptom and easily becomes a source of new errors: stale data looks worse than a slow page.

Even before development, at the briefing, we ask how many people will work at the same time and how much data is expected in a year. Load shapes the architecture: what works for 10 users breaks at 500.

What optimisation cannot fix

A wrong data model. If relationships were designed badly at the technical specification stage, no index will save the day: you will need refactoring and a migration.

What the business gets

  • A before-and-after measurement: which pages and operations were slow and what changed.
  • Fast screens for the work staff do every day.
  • Server costs that grow with the business, not with mistakes in queries.
  • A list of places where the problem is the data model rather than performance, with a refactoring estimate.

Cost and what it depends on

The track costs a fixed $4700 and covers database optimisation, caching, algorithm optimisation and server performance. At the consultation we identify which operations are critical to the business and what load the system must handle. If another team wrote the system, we start with an audit of the existing system: $800 for a written report, or $2400 with an early prototype of the change.

What breaks if you cut corners

  • The budget goes on a bigger server, and the slow query stays slow.
  • A cache added without clearing rules shows customers old prices and stock levels.
  • The system copes while data is small and falls over on the busiest day of the season.
  • Optimisation is done by guesswork and fixes the wrong bottleneck.

Examples from our work

An example from our own house. An audit of our blog showed that 36 posts had text over 100,000 characters long, because images were embedded straight into the article text as encoded strings. Pages like that are almost impossible to index because of their load speed. A more powerful server would not have helped: the problem was how the data was stored. We moved the images into separate files, and today none of the blog's 1,700 articles contains an embedded image.

In projects with real-time stream processing, performance is especially visible. If processing cannot keep up with the stream, data piles up in queues, latency grows, and within a minute a conversation runs ten seconds behind. A well-built system knows when to drop the excess instead of hoarding it.

Where to go next

Related tracks: caching and offline mode and refactoring. The map is in the app development strategy.

Discuss performance.

If your system is slow and buying a bigger server helps less and less, book a free consultation with an architect. We will work out where to look for the cause, and after the consultation we will send a price and timeline broken down by stage.

Frequently asked questions

01Why is an app slow even on a powerful server?

Most often the cause is not capacity but how the system handles data: a query without an index reads the whole table, database calls repeat inside a loop, a heavy report is recalculated on every request, or images are stored inside the text of records. A powerful server only postpones the problem. The cause is found by measuring the slowest and most frequent operations.

02How much does server and database optimisation cost?

In the Ingello development strategy, the Server Optimisation track costs a fixed $4700. It covers database optimisation (indexes), caching, algorithm optimisation and server performance improvement. If another team wrote the system, an audit of the existing system comes first: $800 for a written report, or $2400 with an early prototype of the change.

03Is it better to upgrade the server or optimise the code?

Measure first. If the server hits CPU or memory limits with normal queries, adding resources is justified. If particular queries or algorithms turn out to be slow, new hardware will achieve almost nothing while the bill goes up. Optimising indexes, queries and algorithms usually helps exactly where buying more capacity has stopped working.

04When does caching hurt an app instead of speeding it up?

When a cache is added first, without measurement and without clearing rules. It hides the slow query but starts showing stale data: old prices, stock levels, order statuses. So caching comes last, only for heavy and rarely changing data, and for each cache the event that clears it is defined in advance. In the Ingello stack this is done with Memcached.

05Can a system be sped up if its database is poorly designed?

Partly. Indexes and caching smooth over the symptoms, but they cannot fix a wrong data model: if relationships between entities are badly designed, queries will be slow on any hardware. In that case optimisation pinpoints the problem areas, and the cure is refactoring and data migration. That is a separate growth track with its own estimate.