Business
Airbyte is an open-source data pipeline platform
If you’re working for a company that handles a ton of data, chances are your company is constantly moving data from applications, APIs and databases and sending it to a data warehouse or a data lake. There are a few well-known platforms to extract, load and transform data, such as Fivetran and Stitchdata.
Airbyte is building an open-source data pipeline platform. An open-source alternative to Fivetran or Stitchdata is interesting because building and maintaining connectors require a ton of resources. If you’re just starting out with your data pipeline and you don’t have a big team, it creates a barrier to entry.
Sure, existing data pipeline platforms already provide dozens of integrations with popular sources and destinations. Popular sources include Salesforce, Stripe, Marketo, SendGrid, etc. Popular destinations include Redshift, Snowflake and BigQuery.
But there are a ton of small services that aren’t supported on traditional data pipeline platforms. If you can’t import all your data, you may only have a partial picture of your business.
A lot of companies end up building custom connectors for their applications. With this open-source approach, Airbyte thinks it can foster a community of users to build and maintain thousands of open-source connectors. When a company builds a custom connector, another company can use it, improve it and give back to the community.
“The open-source approach is the right approach,” co-founder and CEO Michel Tricot told me. “If somebody using a connector realizes that it’s broken, they can fix it for the entire community.”
The team behind Airbyte originally started with a product focused on marketing data. They attended Y Combinator and later pivoted because of the coronavirus outbreak. “In a COVID world, there’s no marketing budget,” co-founder John Lafleur told me.
Airbyte connectors run in Docker containers, which means they all operate independently from each other. You can schedule updates, refresh your connectors manually and monitor them individually.
Right now, there are 46 connectors — Airbyte is currently working on certifying them to make sure they’re production-ready. 250 companies are currently using Airbyte.
-
Entertainment7 days ago
Explainer: Age-verification bills for porn and social media
-
Entertainment6 days ago
If TikTok is banned in the U.S., this is what it will look like for everyone else
-
Entertainment6 days ago
‘Night Call’ review: A bad day on the job makes for a superb action movie
-
Entertainment6 days ago
How ‘Grand Theft Hamlet’ evolved from lockdown escape to Shakespearean success
-
Entertainment6 days ago
‘September 5’ review: a blinkered, noncommittal thriller about an Olympic hostage crisis
-
Entertainment6 days ago
‘Back in Action’ review: Cameron Diaz and Jamie Foxx team up for Gen X action-comedy
-
Entertainment6 days ago
‘One of Them Days’ review: Keke Palmer and SZA are friendship goals
-
Entertainment3 days ago
‘The Brutalist’ AI backlash, explained