big data

  • Importing and exporting data into HDFS and Hive using Sqoop. 

big data/hadoop developer

  • Analysis/tracking of captured project requirements using Rally tool.
  • Involved in developing Hive scripts to parse the raw data, populate staging tables and store the refined data in partitioned tables in the Hive.
  • Created Spark Jobs to transform the data and used Spark SQL to created data frames for querying. 
  • Hands-on experience in using Hive partitioning, bucketing and execute different types of joins on Hive tables and implementing Hive SerDes like JSON and Avro.
  • Worked on both External and Managed HIVE tables for optimized performance.
  • Worked extensively with HIVE DDLs and Hive Query language (HQLs) and implemented business logic using Hive UDF’s to perform ad-hoc queries on structured data.
  • Involved in creating Hive tables for Extracting, Transforming and Loading the data and writing hive queries that will run internally in Map Reduce way.