Data Integration Assessment
Data Integration Assessment Requirements
- Is ETL infrastructure used efficiently inside organization?
Dev-Test-Prod Env, HA/DR…
- What are ETL Best Practices?
Development, Performance, Deployment, Architecture, Standards
- How to detect performance bottlenecks?
- How to overcome detected performance issues?
- How to manage and recover failed ETL jobs?
- How to detect unused objects and Developments?
- Which user should have which privilege?
- How to manage User Groups and Roles?
- How to check ETL requirements in daily processes?
Repository queries, Command line capabilities
Data Assessment Framework
Data Integration Assessment Approach
Assess Capabilities – Scoring
- Understand current ETL architecture
- What are the environments such as Dev, Test and Prod and their Resources?
- What are source and target systems?
- Technical Architecture?
- How is deployment working?
- User Management, Groups and Roles
- Interview/Questionnaire with Developers
- Challenges in ETL development
- Identify Developer behaviour
- Measure Server Performance
- Identify ETL Jobs with low performance
- Defining ETL Standards for Naming, Development, Deployment
- Checklist for ETL Standards
- Assess and Strategies to Improve Performance
- ETL Jobs with low performance (Small Workshop to check results)
- Server Performance
- Comparing current situation with best practices
- Repository & Domain Management
- Team Based Development and Versioning
- Security and User Management
- Assess Deployment problems
- Creating an Assessment Report
- Current Situation
- Data Integration Assessment Score
- Architecture Recommendations
- Performance Findings & Recommendations
- Standards Best Practices
- HA/DR Recommendations
- Deployment Recommendations
- Presentation for Assessment Results