unthinkable
Case study · Drewry · maritime research and consulting, 3,000+ clients

Drewry's clients get freight rate insight from processing that now takes hours, not a week.

Drewry, serving over 3,000 clients in 100+ countries, wanted shipping stakeholders to act on volatile container freight rates with timely insight from its Container Freight Intelligence Portal.

Fleet & logistics (transport, dispatch, analytics)

What changed

97.02%improvement in data processing time (as reported)
4-5 hoursto process 150,000 records, down from a week (as reported)
7xdata scalability (as reported)
20xfaster freight costing calculations (as reported)
What stood in the way

What stood in the way.

What stood in the way
  • Large EKS node groups for the portal's ETL pipelines were provisioned by hand and sat idle much of the time, wasting cloud spend.
  • Freight rate data was large and complex, and slow processing held back the insight clients needed.
What changed
  • 97.02%improvement in data processing time (as reported)
  • 4-5 hoursto process 150,000 records, down from a week (as reported)
  • 7xdata scalability (as reported)
  • 20xfaster freight costing calculations (as reported)
What we built

What we built.

  • CloudFormation templates, versioned in Amazon S3, that launch EKS node groups in the sizes each job needs.
  • Apache Airflow DAGs, linked through Boto3, that create the stacks before a heavy run and delete them when it ends.
  • Pipelines split into parallel runs on high-capacity nodes that exist only for the once- or twice-a-month processing.
  • Airflow itself running on Amazon EKS, and CI/CD with AWS CodeCommit and Jenkins on EC2.
  • AWS CloudFormation
  • Amazon EKS
  • Apache Airflow
  • Boto3
  • Amazon S3
  • Jenkins
  • AWS CodeCommit
What it shows

What this work shows about us.

We know freight data moves in cycles

Rate processing peaks once or twice a month, so the infrastructure appears for the run and disappears after it.

Speed and cost together

Parallel runs on large nodes made processing fast; deleting them afterwards kept the cost down.

Want similar on-demand cloud for your data portal? Tell us what you want to change. Our AI assistant, Ask Unthinkable, helps you define your requirements, then connects you with the right team.

Start your plan ↗
If we built it today

If we built it today, with AI.

This work came before today's AI. If Drewry asked us now, this is what we would add, step by step.

Quote

An agent would answer a client's question on a lane's freight rate from the latest benchmark in the portal, with the data behind it. Analysts still own every published figure.

A delay

An agent would watch each monthly run, spot a stalled or failed pipeline step and restart it or call the engineer on duty. Engineers still approve changes to the pipelines.

Ask Unthinkable

Want this for your business?

Tell us what you run or what you're building. Our AI assistant, Ask Unthinkable, helps you define your requirements, then connects you with a senior AI engineer.

Send us a note instead

Required

Thanks. A senior AI engineer will reply to you.
u
Ask UnthinkableClarity first, then the right team
Your conversation

An AI agent. This prototype uses prepared answers and sends nothing.