🧵 THREAD: Why we spent 6 brutal hours internally fighting over blockchain data streaming architecture at Ormi (and why it matters for YOUR crypto data) 🚀🔗💸

1/ At Ormi, we just wrapped a 6-HOUR internal debate about our blockchain data pipeline. Why so intense? Because how we stream, adapt and transform crypto data directly impacts how we serve YOU, our clients. 💯

2/ The state of modern data streaming for blockchain is BRUTAL. 😤 So much critical infrastructure sits behind paywalls, with companies acting like these problems are "solved" when they're just gatekept. 🔒💰

3/ Sure, we could use @ApacheKafka. It's battle-tested and reliable. But is 2010's tech stack really the foundation we want for the future of blockchain data processing? We thought we could do better. 🤔

4/ At Ormi, we're committed to Open Source. Not just because it's trendy, but because we believe in building sustainable, extendable infrastructure that doesn't lock clients into vendor relationships. 🔓🛠️

5/ We strongly considered @RedpandaData, but the proprietary nature gave us pause. After intense debate (and yes, some raised voices 🗣️), we're taking a completely different approach. 🔄 Our decision: We're building Ormi's blockchain data pipeline on Bento - a true open-source

6/ When @RedpandaData acquired Benthos (now rebranded as "Redpanda Connect"), they made aggressive moves in just 12 hours: - Prohibited using the term "Benthos" 📝 - Controlled community discussions 🗨️ - Started making critical integrations proprietary 🔐

7/ The community forked the project as "Bento" and committed to keeping it truly open source forever. Bento isn't just a principled choice - it's POWERFUL. 💪 It connects seamlessly to Postgres, ClickHouse, AWS S3, and GCP storage. The SQL integration lets us query databases

8/ Yes, building our own ETL would be a nightmare - monitoring, tracing, CDC, K8s integration... 😱 The engineering hours would be astronomical. Bento gives us enterprise-grade capability without enterprise-grade price tags. 💰

9/ Blockchain data shouldn't be locked behind proprietary systems. At Ormi, we're putting our engineering hours where our values are - building on truly open infrastructure that will stand the test of time. ⏳🔨

10/ We're even contributing back to the Bento project, potentially enhancing batch metadata and file sorting capabilities. Open source isn't just consumption - it's contribution! 🌱💻

11/ The decision wasn't easy, but the best technical choices rarely are. Six hours of heated debate led to a solution that will serve our clients for years to come. 📈🔮

12/ Want to learn more about how we're using Bento to revolutionize blockchain data streaming? Follow us or drop a comment below. We're building the future of crypto data infrastructure, one stream at a time. 🔄🔗📊