Drewry's clients get freight rate insight from processing that now takes hours, not a week.
Drewry, serving over 3,000 clients in 100+ countries, wanted shipping stakeholders to act on volatile container freight rates with timely insight from its Container Freight Intelligence Portal.
What changed
What stood in the way.
- Large EKS node groups for the portal's ETL pipelines were provisioned by hand and sat idle much of the time, wasting cloud spend.
- Freight rate data was large and complex, and slow processing held back the insight clients needed.
- 97.02%improvement in data processing time (as reported)
- 4-5 hoursto process 150,000 records, down from a week (as reported)
- 7xdata scalability (as reported)
- 20xfaster freight costing calculations (as reported)
What we built.
- CloudFormation templates, versioned in Amazon S3, that launch EKS node groups in the sizes each job needs.
- Apache Airflow DAGs, linked through Boto3, that create the stacks before a heavy run and delete them when it ends.
- Pipelines split into parallel runs on high-capacity nodes that exist only for the once- or twice-a-month processing.
- Airflow itself running on Amazon EKS, and CI/CD with AWS CodeCommit and Jenkins on EC2.
- AWS CloudFormation
- Amazon EKS
- Apache Airflow
- Boto3
- Amazon S3
- Jenkins
- AWS CodeCommit
What this work shows about us.
Rate processing peaks once or twice a month, so the infrastructure appears for the run and disappears after it.
Parallel runs on large nodes made processing fast; deleting them afterwards kept the cost down.
Want similar on-demand cloud for your data portal? Tell us what you want to change. Our AI assistant, Ask Unthinkable, helps you define your requirements, then connects you with the right team.
Start your plan ↗If we built it today, with AI.
This work came before today's AI. If Drewry asked us now, this is what we would add, step by step.
An agent would answer a client's question on a lane's freight rate from the latest benchmark in the portal, with the data behind it. Analysts still own every published figure.
An agent would watch each monthly run, spot a stalled or failed pipeline step and restart it or call the engineer on duty. Engineers still approve changes to the pipelines.
Where this work fits.
- Logistics & fleetsYour logistics business is meant to deliver every load on time, without shippers wondering where it is.
- Fleet & logistics (transport, dispatch, analytics)Every load delivered on time, and shippers and dispatchers hear about a delay before they have to ask.
- Area OwlTravellers find nearby hotels, restaurants and homestays in a few clicks, and local businesses reach them directly.
- A leading travel agency in GreeceTravellers plan and book flights, hotels and whole trips on one portal that shows what is really available.
Facts and figures from the project records; nothing added.
Want this for your business?
Tell us what you run or what you're building. Our AI assistant, Ask Unthinkable, helps you define your requirements, then connects you with a senior AI engineer.
Send us a note instead
An AI agent. This prototype uses prepared answers and sends nothing.
We use essential cookies to run this site. With your consent we also use analytics cookies to see which pages help. Privacy policy