Bigdata Engineer Bigdata Engineer …

Cognizant
in Melbourne, Victoria, Australia
Permanent, Full time
Be the first to apply
Competitive
Cognizant
in Melbourne, Victoria, Australia
Permanent, Full time
Be the first to apply
Competitive
Bigdata Engineer
Job Description:
  • Hands on experience on Spark (Scala Spark) -
  • Spark RDD API
  • Spark SQL Data frame API
  • Spark Streaming API
  • Hands on programming skills on - Scala, Python, Shell Scripting
  • Should have exposure to RDBMS, No-SQL like HBase etc...
  • Hands on experience on Versioning system like - Git, BitBucket
  • Sound knowledge on Spark Query tuning and performance optimization.
  • Experience working on Cloudera, HDFS
  • Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency,
  • and consensus)
  • Advantage on having knowledge on Real-time tool - Kafka
  • Advantage on having knowledge on Cloud Platforms, i.e Google Clod Platform, AWS or Azure
  • Added advantage to have awareness of Agile Methodology and tools like JIRA
  • Leadership, drive outcomes with upstream teams, engage with peer engineers, lead them to solutions that enable connectivity, be it code or system changes on their side, or simply new functional accounts.
  • Responsible to design and build analytics project which include task like data ingestion, data transformation, data quality, data engineering and data governance in Big Data.
  • Design, architecture and implementation of complex information system using various Big Data components, Batch(ETL/ELT) and streaming APIs, complex data model design.
  • Design and Create Scala/Spark jobs for data transformation and aggregation
  • Responsible to process huge volume of structured and semi structured data and ingest data in data lake.
  • Responsible for SDLC, including analysis, design, development, implementation, support and enhancement.
  • Understand banking remediation issues and provided the solutions to handle in big data
  • Work on defect fixes, enhancement and production support of different applications.
  • Develop SQL queries to perform data analysis and data validation.
  • Develop migration scripts to deploy the code from one environment to another environment.
  • Technical support to business system analyst and customers to assist them with resolution of business incidents/tickets
  • Researching new tools, technologies and practices for improving system efficiency.
Salary Range: >$100,000

Cognizant logo
More Jobs Like This
See more jobs
Close
Loading...