Senior Data Engineer X2
Description Data Engineer IV At American Family Insurance, we believe people are an organization’s most valuable asset, and their ideas and experiences matter. From our CEO to our agency force, we’re committed to growing a diverse and inclusive culture that empowers innovation that will inspire, protect, and restore our customers’ dreams in ways never imagined. American Family Insurance is driven by our customers and employees. That’s why we provide more than just a job – we provide opportunity. Whether you’re already part of our team in search of a new challenge or new to our company and ready for what’s next, you’re in the right place. Every dream is a journey that starts with a single step. Start your journey right here. Join our team. Bring your dreams.
Job Description
As a Data Engineer IV, you’ll work on collecting, storing, processing and building Business Intelligence and Analytics applications within our big data platform. Presently, our team is constructing an enterprise data lake to enable analysts and scientists to self-service data at scale across American Family’s operating companies. We’re leveraging open source technologies like Spark, Python, Hadoop, and cloud native tools to curate high-quality data sets. You’ll also be responsible for integrating these applications with the architecture used across the organization. Adjacent responsibilities include establishing best practices with respect to data integration, data visualization, schema design, performance and reliability of data processing systems, supporting data quality, and enabling convenient access to data for our scientists and business users. Job Level Summary Recognized as an expert within the company and requires in-depth and/or breadth of expertise in own job discipline and broad knowledge of other job disciplines within the organization function Solves unique problems that have a broad impact on the business Contributes to the development of organization functional strategy Progression to this level is typically restricted on the basis of business requirements Primary Accountabilities Perform exploratory data analysis to determine which questions can be answered effectively with given data. Ability to analyze new (possibly unstructured) data sources to determine what additional value they may bring and how to effectively make use of them. Design and develop highly scalable and extensible data pipelines from internal and external sources. Work on cross-functional teams to design, develop, and deploy data-driven applications and products, particularly within the space of data science. Lead in prototyping complex data source processing and emerging technologies involving data ingestion and transformation, distributed file systems, databases and frameworks. Work with our data science team on recommending and applying improvements to their machine learning algorithms and platforms. Design, build, and maintain tools to increase the productivity of application development and client facing teams. Partner with business analyst to define, develop, and automate data quality checks. Review developed solutions to solve specific business problems. Optimize queries, data models, and storage formats to support common usage patterns. Design and develop big data applications and data visualization tools. Code structure moves beyond procedural ad-hoc workflows and exhibits modularity and comprehensive test cases. Develops code at a high-level of abstraction – writes, maintains, and reuses utilities/libraries across the enterprise. Influences strategy related to processes and workflows across their division, as well as the entire enterprise. Participates in mentoring junior colleagues, as well as other teams throughout the organization Specialized Knowledge & Skills Requirements Demonstrated experience providing customer-driven solutions, support or service. Significant experience using SQL or NoSQL and experience using a variety of data stores (e.g. RDBMS, analytic database, scalable document stores). Extensive hands-on Python programming experience, with an emphasis towards building ETL workflows and data-driven solutions. Able to employ design patterns and generalize code to address common use cases. Capable of authoring robust, high quality, reusable code and contributing to the enterprise inventory of libraries. Exhibits mastery over big data batch computing tools (e.g. Hadoop or Spark), with demonstrated experience developing distributed data processing solutions. Applied knowledge of cloud computing (AWS, GCP, Azure). Knowledge of open source machine learning toolkits, such as sklearn, SparkML, or H2O. Extensive data understanding and business acumen in the data rich industries like insurance or financial Applied knowledge of data modeling principles (e.g. dimensional modeling and star schemas). Strong understanding of database internals, such as indexes, binary logging, and transactions. Experience using tools for infrastructure-as-code (e.g. Docker, CloudFormation, Terraform, etc.), as well as designing and distributing generic infrastructure templates that support development workflows across the enterprise. Experience with software engineering tools and workflows (i.e. Jenkins, CI/CD, git). Extensive hands-on experience with Linux/Unix – proficiency in navigating in the terminal, configuring environments, accessing remote servers, and developing on the platform. Education and Licenses Bachelor’s degree in computer science or related field, or equivalent combination of education and experience. Apply tot his job Apply To this Job