“Tech Connects,” Dice’s podcast, digs into the tech hiring, recruiting, and career topics that matter to you. Subscribe on ACast, Spotify, Apple Podcasts, iHeartRadio, Amazon Podcasts, and YouTube!
Data is the lifeblood of organizations everywhere. On the latest episode of ‘Tech Connects,’ we’re speaking with Steven Hillion, SVP of data and A.I. at Astronomer, a company that helps clients manage their data pipelines. Astronomer is a commercial developer of Airflow, an open-source platform originally developed at Airbnb as a way for that tech giant to manage all of its data platforms and data pipelines.
During our chat, Hillion provides some key insights into so many of the data issues impacting companies today, from verifying the quality of data to fine-tuning the large language models (LLMs) that power the current generation of generative A.I. products. Let’s listen in!
If you’re someone who works with data—and increasingly, everyone’s working with data—you may draw some useful information from this episode. Here are some quick takeaways from the discussion:
First, it’s important to define ‘data quality.’ Is there consistency in your data sets? Is everyone comfortable with the sources, metrics, and outputs? Everyone in your organization should have confidence in your data and the insights you’re producing from it. Fortunately, there’s a variety of tools that allow you to manage and evaluate data quality.
Second, when it comes to A.I., it might be best to proceed with caution. Embrace the technology, sure, but also keep an eye on what others are doing. It’s early days for A.I., which means it’s difficult for everyone to find truly the right way forward. Within a year or two, Hillion thinks there will be reference architectures and implementations that will establish some guardrails for A.I. development; but until then, it’s important to be careful as you build and test your own A.I.-based solutions.
Third, if you’re interested in data science as a profession, you’ll need to learn a core group of skills, including (but definitely not limited to) Python—which Steven calls the lingua franca of a data scientist—SQL, machine learning, and statistics.
We covered a whole lot of other topics during the episode, of course, so give it a re-listen if there was something you missed. We’ll see you next time—and remember, Dice is your best resource to find the tech talent you need to fill your open roles, and for technologists, the best place to grow your tech career.
