How to Save 85% on 25 Billion Events
Canva saved a lot of money by updating their event architecture on AWS. Let's know how they did it
Victor Garcia
August 11, 2024
What happened this week:
Canva brutally reducing its costs on event processing at AWS
Great articles about Reserved Instances
Mention of Honour to FinOps & GreenOps synergy
Job Offers
Professional Spotlight with two of the most valuable professionals in the FinOps Sector
It’s been a heavy week, so a lot of value is coming. Brace yourself:
Big Tech
Canva 85% Cost Savings on 25 Billion Events
In today’s issue, we'll see the latest choices made by Canva to manage its massive event data-processing needs.
Switched to Amazon KDS
Chose Amazon Kinesis Data Streams over SNS+SQS → Saved 85% on costs.
Managing 25 Billion Events
Powering features like personalization and usage stats → Data crucial for A/B testing.
Cost-Efficient Strategies
Used batching and zstd compression for savings → Achieved $600k annual savings with compression.
Handling High Throughput
Implemented fallback logic using SQS → Maintained p99 latency below 20ms.
Schema Management
Used Protocol Buffers for event definitions → Created Datumgen for compatibility and code generation.
Canva's example highlights significant advancements in cost management and performance optimization for data-heavy applications.
Cost Optimization
Optimizing Cloud Costs: Strategies and Challenges Article Series by Shahar Raz
Shahar M. Raz recently shared some insightful articles on cloud cost optimization.
Optimizing cloud costs needs careful planning and regular review.
Regular, small steps lead to better long-term savings.
Companies face obstacles with Reserved Instances.
Many don't leverage them effectively.
Offers practical steps and real-life tips.
Helps overcome barriers to optimization.
Discusses Reserved and Spot Instances.
Shows how to balance different instance options.
These articles are a must-read for anyone looking to reduce their cloud costs!
Reserved Instances
Reserved Instances: Cost Savers or Accountability Killers?
Anderson Oliveira discusses the tricky balance of Reserved Instances (RIs) and team accountability. Here’s what you need to know:
RIs are Great for Savings. RIs can cut 50%-70% of Virtual Machine costs. They require a 1 to 3-year commitment.
The Accountability Issue
Engineering teams juggle many tasks, including cost optimization. But costs can change unexpectedly due to RI effects, confusing everyone.
RIs are easy savings but require continued FinOps efforts. Teams need clear cost reports linking to their usage. As you may already know, RIs affect prices beyond Engineering's control.
Achieving Stable Pricing: Aim for consistent pricing to show true usage costs. Use "blended" or "full" pricing to maintain clarity.
RIs can save money, but they complicate cost management. FinOps teams must find a balance.
Event
FinOps Excellence Summit Summary
The FinOps Excellence Summit wrapped up with key discussions on cutting cloud costs and optimizing operations.
Missed it? Recordings are available on demand. Sessions include:
Fix It and Make It Budget Friendly: Balance cost efficiency with agility. Learn cost-consciousness early in development. Quick wins in cost optimization were discussed.
Don't Be a Cloud Cost Chump: Explore AI in FinOps.
Weigh the benefits and costs of AI. Strategies for tracking AI initiatives shared.
Is My Cloud Bill Trying to Tell Me Something?
Decode cloud bills to find cost-saving patterns. Learn to make informed financial decisions.
Recordings are available here: Link.
Mention of Honour Special
Deserved for recent discoveries that are inspiring for the FinOps community.
🎖️ Mention of Honour
GreenOps and FinOps: Boost Profit and Sustainability
A great article by Graziano Casto on CNCF where he explores the connections between GreenOps and FinOps, showing how integrating these methodologies can help organizations achieve business goals while managing their environmental impact.
Synergy Between GreenOps and FinOps
Shared Goals: Both aim to optimize cloud usage.
Balanced Costs and Emissions: Integrate to reduce both expenses and carbon footprint.
Same Practices: Strategies like right-sizing and storage tiering apply to both.
Using Tools: Tools like kube-green automate shutting down unused resources.
Real-World Application
Software Delivery Pipelines: Reduce resource usage in development environments.
Use of Kube-Green: Automated resource management during off-peak hours.
Significant Impact: Lower emissions by 38% and save costs.
Sustainable and Cost-Effective: Achieve efficiency while protecting the environment.
Combining FinOps and GreenOps makes cloud operations both economically and environmentally friendly.
Jobs
👩🏭 FinOps Job offers
I'll find both Permanent links (i.e. FinOps Job site) and Offers on the internet.
Professional Spotlight
Frank Contrepois & Stephen Old
An inspiration in the field, and to this newsletter. Frank & Stephen, the FinOps Guys, are doing an amazing job in the community.
They share a podcast adding great value with new findings and interviews.
You should listen to it!
Bonus
🏆️ Quick Wins
For those seeking more content, this section will provide additional resources.
Announcements
As has been seen on LinkedIn a lot, I just mention it: AWS Discontinuing tons of services
Azure's Carbon Optimization tool previews August 2024.
The Forrester Wave: Cloud Cost Management And Optimization Solutions, Q3 2024
Articles
Ingesting SageMaker prices into FOCUS Data Lake by FinOps Scribbler (dear Benjamin!)
FinOps Foundation Guide on Cost Estimation of AI Workloads
Podcasts
Feedback
🗳️ Poll of the Week
🎁 Rate our Referral Program 🎁 |
That’s all for this week. See you next Sunday!
If you like the content, you can subscribe to receive it every week.
Let’s Collaborate!
Connect on Linkedin: I share valuable free content there. I am open to chatting with anybody interested in Cloud & FinOps.
Set up a Free Discovery Call: Let’s discuss how we can collaborate.
Sponsor this newsletter: Promote your company in this newsletter and reach the Cloud audience that wants to maximize business value and reduce costs.
PS1: Know someone who’ll enjoy this newsletter? Please forward it. You’ll make FinOps grow bigger.
PS2: Was this email forwarded to you? Subscribe here and get all editions!
PS3: I’ll do my best to provide you with the best content. However, some of it may slip. If you think something should have been included, please reach out on LinkedIn or REPLY to this email.