big data/hadoop developer
- Analysis/tracking of captured project requirements using Rally tool.
- Involved in developing Hive scripts to parse the raw data, populate staging tables and store the refined data in partitioned tables in the Hive.
- Created Spark Jobs to transform the data and used Spark SQL to created data frames for querying.
- Hands-on experience in using Hive partitioning, bucketing and execute different types of joins on Hive tables and implementing Hive SerDes like JSON and Avro.
- Worked on both External and Managed HIVE tables for optimized performance.
- Worked extensively with HIVE DDLs and Hive Query language (HQLs) and implemented business logic using Hive UDF’s to perform ad-hoc queries on structured data.
- Involved in creating Hive tables for Extracting, Transforming and Loading the data and writing hive queries that will run internally in Map Reduce way.
- Importing and exporting data into HDFS and Hive using Sqoop.