OpenArc - Empowering Your Career. As a leading IT staffing firm, we are dedicated to connecting talented professionals with your ideal opportunities. We are currently seeking a qualified Data Engineer  to join our client’s organization and contribute to their ongoing success. 

The Engineer IV, Data is an experienced professional who participates in creating and extending product optimizations. This person can apply best practices to take data from multiple sources, build and manage a pipeline to serve advanced analytics including dashboards, reporting, and even ML.

Responsibilities:

• Lead the solution  design for a diverse  range  of complex problems  with  a collaborative  approach with team members on a single scrum team

• Provide technical leadership to agile teams–onshore and offshore: Mentor other team members ranging from junior engineers to new team members.

• Implement data pipelines on a scrum team to deliver new data sources and features for machine learning.

• Implement data quality checks, classification, and data lineage.

• Write and validate test cases for other team member’s code.

• Implement unit tests to cover the new features being created.

• Augment the CI/CD pipeline to deliver new features to production.

• Approve pull requests after a through code review or require changes to better solve the problem or adhere to standards.

• Provide engineering support for escalations from the 24/7 Technical Assistance Center when production downtime occurs.

Requirements:

• Advanced SQL skills with an emphasis on being able to review and understand data via ER Diagrams and exploratory queries.

• Deep development experience of distributed/scalable systems and high-volume transaction applications, architecting big data projects. 

• Ability to conduct unit testing using automated frameworks.

• Ability to convey ideas and integrate with external systems.

• Expert in using Big data technologies like, Apache Kafka, Apache Spark, Real Time streaming, Structured Streaming, Delta lake.

• Expert in Agile development practices and Software Quality Assurance process. Comfortable in shipping new features in complex environment.

• Uses independent judgement to accomplish objectives.

• Excellent analytical and problem-solving skills.

• Focus on development, architectural consultancy, teamwork

• Energetic, motivated self-starter that is eager to excel with excellent inter-personal skills.

• Expert in knowing a balance driving the right architecture but realizing the realities of having customers and the need to ship software.

• Able to establish great relationships with managers and team, communicate properly on time, able to ensure the team work efficiently.

• Hands-on development experience with distributed/scalable systems and high-volume transaction applications

• Experience in data profiling, management, and ETL methodology 

• Hands-on programming experience in Scala, Python or other object-oriented programming languages. 

• Experience in creating architecture diagrams and models.

At OpenArc, we prioritize your career success and strive to build exceptional technical teams for our clients. By understanding your experience and aspirations, we ensure to present you with rewarding and fulfilling opportunities. 

As an employee of OpenArc and our clients, you will be eligible to participate in a comprehensive benefits package.