Big Data Developer Resume Examples

big data developer

  • Involved in analysis of requirements and business rules based on given documentation and work closely with tech leads and Business analysts in understanding the current system.
  • Analyze the data coming from different resources to know its schema and functionality.
  •   Use Sqoop scripts to ingest data from different RDBMS sources into Hadoop Cluster (HDFS) and created Hive tables, partitions, data loading into hive tables, etc.
  •    Worked on several functions in Scala Library to build Spark Applications, Spark SQL, RDDs -Transformations, Actions, data frames and pushed the results to the HDFS.
  • Developed Hive programs to parse the raw data, populate staging tables and to store the redefined data in partitioned tables in the EDW.
  • Created Hive Queries and Scala applications that helped marketing analysts to spot emerging trends by comparing fresh data with EDW reference tables and historic metrics. 

big data developer

  • implemented Python and Go scripts REST API’s to connect different file systems (HDFS, SFTP and Aws S3), Database systems (ORACLE, PostgreSQL, MySQL, Cassandra and Mongo DB) and external systems (SAP, Salesforce and Kinaxis) for ETL process.
  • Implemented to upload and process large scale of files with in short time using Scala and Spark RDD/DF technologies. 
  • Design, Modified, Improved Python, JavaScript by code refactor. 
  • Responsible for Design and Develop the Scala and Node based technologies.
  • Worked closely with Business and Technical Design architects to understand the flow.
  • Used docker/ kubernetes mechanism to run all services of product.
  • Automated data extract and load into different systems under SCM with high security.

big data developer

  • Modified the existing code as per the requirements.
  • Decreased the run time of the jobs by performing performance tuning in the hive scripts.
  • Monitored jobs in production ensuring that there are no failure.
  • Fixed failed jobs and bugs in the application.
  • Automated the logs of every feed received by the application for the better understanding of the downstream team.
  • Co-authored design document for the new requirement of the existing application. 

big data developer

  • Client : Intel 
  • Environment : LINUX Software 
  • Development Model : Agile Model 
  • Implemented automated mail alert system for ETL flow status.

big data developer

  • Gain hands-on experience on polyglot development culture.
  • Write Unit and Integration tests to ensure end to end code coverage.
  • Recommended architectural improvements, design solutions and integration solutions. (Optimise workflows, choosing better platforms, migrating CI Pipelines)
  • Design RBAC system and developed REST API’s by NodeJs, MySQL database.

big data developer

  • Worked as Offshore lead to gather business requirements and guided the team
  •  Implemented spring boot micro services to process the messages into the Kafka cluster setup
  • Used Spark SQL on data frames to access mysql tables into spark for faster processing of data.
  • Configured Spark streaming to receive real time data from the Kafka and store the stream data to HDFS.
  • Used Spark for interactive queries, processing of streaming data and integration with popular NoSQL database for huge volume of data.
  • In preprocessing phase of data extraction, used Spark to remove all the missing data for transforming of data to create new features.
  •  Improved Spark performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark SQL, Data Frame, PairRDD’s, Spark YARN

big data developer

  • Gathered requirements from the Onsite lead
  • Involved in all the team meetings & meeting with the clients.
  • Worked on creating Hive table definitions.
  • Worked on getting the data from the files and loading them to the Hive external tables.
  • Enriching the data in Hive tables by joining different tables and creating a final view for the enriched data.
  • Involved in the Development, testing and validation of the entire code related to Hive, NIFI, Shell Scripting
  • Worked with the Hadoop admin for the deployment of the project.

big data developer

  • Analysis, Design and Development of complex business applications in Apache Spark with Scala. 
  • Orchestration of a vast set of applications involving Spark applications, Web Services, Python and Shell Scripts using Apache Oozie and NIFI. 
  • Design the end to end flow using Apache NIFI which helps in data routing, transformation, and system mediation logic.
  • Responsible for Development of a scripting based framework which is used as the base for Orchestration in the complete project. 
  • Work with the Architect directly to manage large, complex design projects for corporate clients. 
  • Grooming the next generation of developers to understand the architecture and functioning of the project. 

senior big data developer

  • Understand ,design and implement requirement to meet the client expectation.
  • Understand the raw data ,design and develop spark applications to process,store and visualise the data.
  • Troubleshoot time and memory consuming spark applications and optimise. 
  • Used Agile (SCRUM) methodologies for application development.

big data developer

  • Data Migration from SqlServer to HDP 
  • Played a vital role in building the enterprise data lake. 
  • Hive and Sqoop were used for report generation on the Hadoop platform.
  • Responsible for both development of new technology in the Hadoop stack & administration of the existing clusters.
  • Worked on Referral programs (Digidhan & Bhim Referral schemes to name a few). 
  • Worked on designing interactive analytics architecture on HDP.Architecture is based on lambda architecture.Spark was used for building pipeline & as a processing tool.
  • Hbase as a storage layer & Apache phoenix as a sql layer for interactive analysis on most granular data