Data Engineer for Alvia Systems Inc
Project scope
Categories
Data visualization Data analysisSkills
lifecycle management environmental science data storage azure data lake microsoft azure data engineering environmental studies azure data factory azure databricks cloud technologiesThis project aims for students to understand and implement cloud-based solutions for managing geospatial data related to environmental studies, with a focus on scalability, performance, and security. The objective is to use Azure cloud services to store, process, and optimize geospatial data for easy access and analysis. By the end of the project, students will have learned how to leverage Azure cloud technologies for big data challenges, demonstrating their ability to design and implement efficient data storage solutions that are critical for data-driven decision-making in environmental science and beyond.
Azure Environment Setup and Data Storage
- Objective: Configure an Azure cloud environment suitable for storing large volumes of geospatial data.
- Deliverable: A documented setup of Azure storage accounts, Azure SQL Database, and/or Azure Data Lake storage, including configuration settings for security and scalability.
- Filename for Deliverable: Azure_Environment_Setup.pdf
Data Ingestion and Engineering
- Objective: Ingest geospatial data into the Azure environment and apply data engineering practices to optimize for performance and accessibility.
- Deliverable: A report detailing the data ingestion process, data engineering steps taken (such as indexing for geospatial queries), and any Azure services used (e.g., Azure Data Factory, Azure Databricks).
- Filename for Deliverable: Data_Ingestion_Engineering_Report.pdf
Implementing Best Practices for Data Security and Management
- Objective: Apply best practices for data security, compliance, and lifecycle management within the Azure cloud environment.
- Deliverable: A comprehensive guide detailing the security measures, compliance standards adhered to, and data lifecycle management strategies implemented for the geospatial data stored in Azure.
- Filename for Deliverable: Data_Security_Management_Best_Practices.pdf
Support will be provided through access to Azure cloud resources and subscriptions, mentorship from experienced Azure data engineers, and workshops on Azure services, data security, and data engineering best practices. Weekly check-ins will ensure students are on track, allowing for real-time feedback and assistance.
Supported causes
Climate actionAbout the company
Alvia Systems is a climate mitigation company dedicated to enhancing climate risk transparency and enabling site-level resilience to natural disasters. We specialize in utilizing advanced drone technology and ML-driven analysis to provide precise, comprehensive climate risk assessments for properties, aiding in informed decision-making and proactive disaster preparedness.