Business
Airbyte is an open-source data pipeline platform
If you’re working for a company that handles a ton of data, chances are your company is constantly moving data from applications, APIs and databases and sending it to a data warehouse or a data lake. There are a few well-known platforms to extract, load and transform data, such as Fivetran and Stitchdata.
Airbyte is building an open-source data pipeline platform. An open-source alternative to Fivetran or Stitchdata is interesting because building and maintaining connectors require a ton of resources. If you’re just starting out with your data pipeline and you don’t have a big team, it creates a barrier to entry.
Sure, existing data pipeline platforms already provide dozens of integrations with popular sources and destinations. Popular sources include Salesforce, Stripe, Marketo, SendGrid, etc. Popular destinations include Redshift, Snowflake and BigQuery.
But there are a ton of small services that aren’t supported on traditional data pipeline platforms. If you can’t import all your data, you may only have a partial picture of your business.
A lot of companies end up building custom connectors for their applications. With this open-source approach, Airbyte thinks it can foster a community of users to build and maintain thousands of open-source connectors. When a company builds a custom connector, another company can use it, improve it and give back to the community.
“The open-source approach is the right approach,” co-founder and CEO Michel Tricot told me. “If somebody using a connector realizes that it’s broken, they can fix it for the entire community.”
The team behind Airbyte originally started with a product focused on marketing data. They attended Y Combinator and later pivoted because of the coronavirus outbreak. “In a COVID world, there’s no marketing budget,” co-founder John Lafleur told me.
Airbyte connectors run in Docker containers, which means they all operate independently from each other. You can schedule updates, refresh your connectors manually and monitor them individually.
Right now, there are 46 connectors — Airbyte is currently working on certifying them to make sure they’re production-ready. 250 companies are currently using Airbyte.
-
Entertainment6 days ago
Teen AI companion: How to keep your child safe
-
Entertainment6 days ago
‘Wallace and Gromit: Vengeance Most Fowl’ review: A delightful romp with an anti-AI streak
-
Entertainment5 days ago
‘Dragon Age: The Veilguard’ review: BioWare made a good game again
-
Entertainment5 days ago
Polling 101: Weighting, probability panels, recall votes, and reaching people by mail
-
Entertainment4 days ago
‘Only Murders in the Building’ Season 4 ending explained: Who killed Sazz and why?
-
Entertainment4 days ago
5 Dyson Supersonic dupes worth the hype in 2024
-
Entertainment3 days ago
When will we have 2024 election results online?
-
Entertainment3 days ago
Social media drives toxic fandom. Is there a solution?