Connecting...

Data Engineer

Job details

Location: NSW
Salary: $100K - $140k including super
Discipline:
Reference: Ad-50832
Posted: 4 months ago

Job description

  • Large multinational company
  • CBD location
  • Competitive salary and excellent perks


Your new role

As a Big Data Engineer you will be part of core big data technology and design team. Person would be entrusted to developed solutions/design ideas, identify design ideas to enable the software to meet the acceptance and success criteria. Work with architects/BA to build data component on the Big data environment.

RESPONSIBILITIES:
As a key member of the technical team alongside Engineers, Data Scientists and Data Users, you will be expected to define and contribute at a high-level to many aspects of their collaborative Agile development process:

  • Software design, development, automated testing of new and existing components in an Agile, DevOps and dynamic environment
  • Promoting development standards, code reviews, mentoring, knowledge sharing
  • Product and feature design, scrum story writing
  • Work on Data Engineering pipelines
  • Product support & troubleshooting
  • Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring
  • Liaison with BAs to ensure that requirements are correctly interpreted and implemented. Liaison with Testers to ensure that they understand how requirements have been implemented - so that they can be effectively tested.
  • Participation in regular planning and status meetings. Input to the development process - through the involvement in Sprint reviews and retrospectives. Input into system architecture and design.
  • Peer code reviews.
  • 3rd line support.


CHALLENGES:

  • Integrating with an established, complex project working to tight deadlines
  • Responsible for implementing streaming data platform

What you'll need to succeed

MUST HAVE SKILLS:

  • Experience in Big data tools: Spark, Kafka
  • Must have experience implementing and designed streaming platform Big Data
  • Experienced in Python is must. Python development and design
  • Experience with traditional databases such as Oracle, Teradata, SQL Server
  • Strong understanding of core features of Python and when to use them
  • Experience supporting and working with cross-functional teams in a dynamic environment. Must be able to quickly understand technical and business requirements and can translate them into technical implementations
  • Experience with Agile Development methodologies
  • Experience with data ingestion and transformation
  • Strong communication + someone who can articulate their solutions and drive a leadership approach across the development


Follow links to apply

This job has expired!