Vacancy - Big Data Engineer

Munsoft is currently seeking a Big Data Developer with experience in Nifi, Kafka, Hadoop, Spark and Python within a financial domain. The successful candidate will work in an Agile environment interacting with multiple technology & business areas, as well as external parties in designing and developing next generation big data platforms and applications. The candidate must ensure that data pipelines are scalable, repeatable, and secure, and will facilitate getting data from a variety of different sources, getting it in the right formats, assuring that it adhere to data quality standards, and assuring that downstream users can get that data quickly. The candidate will furthermore be responsible for the infrastructure that provides insight from raw data and handles diverse sources of data seamlessly which will ultimately enable big data and batch/real-time analytical solutions through leveraging emerging technologies.

The successful candidate will assist in setting the strategy around big data & data science within the organisation.

Responsibilities & Requirements

• Design and support end-to-end big data ingestion and transformation pipelines using Apache NiFi
• Having knowledge on advanced analytics with Python or R, PySpark or SparkR will be advantageous
• Translates business concepts to technical implementations to drive alignment and decision making by extracting meaning from data
• Pro-actively identify data quality issues and coordinate with stakeholders to ensure data accuracy
• Helps to create strategic roadmaps specific to the big data and analytics space
• Familiarity with Dev/Ops practices and tools (Salt, Python, Git, JIRA/BitBucket, etc).
• Knowledge of the Accounting/Financial domains
• Experience in designing and implementing always-on Hadoop data lakes to address data storage, partitioning, splitting, file types (Parquet, Avro, ORC) for specific use cases etc.
• Machine Learning know-how of both supervised and unsupervised learning models including Clustering, Regression models, Decision trees, SVM, Naive Bayes etc will be advantageous
• Ability to work in a fast-paced environment and manage multiple parallel projects
• Extensive experience with horizontally scalable and highly available system design and implementation, with focus on performance and resiliency
• Participate in all phases of the data life cycle and independently and collaboratively finalise project requirements, architect solutions and perform data engineering development and supporting duties.

To be considered, your application must be able to match the above criteria. Should you be interested in applying for this role, please send a concise CV to [email protected].