Data Integration Assessment

Data Integration Assessment Requirements

  • Is ETL infrastructure used efficiently inside organization?
    Dev-Test-Prod Env, HA/DR…
  • What are ETL Best Practices?
    Development, Performance, Deployment, Architecture, Standards
  • How to detect performance bottlenecks?
    Server/Database/Network/Development
  • How to overcome detected performance issues?
  • How to manage and recover failed ETL jobs?
  • How to detect unused objects and Developments?
  • Which user should have which privilege?
  • How to manage User Groups and Roles?
  • How to check ETL requirements in daily processes?
    Repository queries, Command line capabilities

Data Assessment Framework

Data Integration Assessment Approach

Assess CapabilitiesScoring

Investigation Capabilities

  • Understand current ETL architecture
    • What are the environments such as Dev, Test and Prod and their Resources?
    • What are source and target systems?
    • Technical Architecture?
    • How is deployment working?
    • User Management, Groups and Roles
  • Interview/Questionnaire with Developers
    • Challenges in ETL development
    • Identify Developer behaviour
  • Measure Server Performance
  • Identify ETL Jobs with low performance

Assess Capabilities

  • Defining ETL Standards for Naming, Development, Deployment
  • Checklist for ETL Standards
  • Assess and Strategies to Improve Performance
    • ETL Jobs with low performance (Small Workshop to check results)
    • Server Performance
  • Comparing current situation with best practices
    • Repository & Domain Management
    • Performance
    • Reusability
    • Team Based Development and Versioning
    • Security and User Management
  • Assess  Deployment problems

Report Capabilities

  • Creating an Assessment Report
    • Current Situation
    • Data Integration Assessment Score
    • Architecture Recommendations
    • Performance Findings & Recommendations
    • Standards Best Practices
    • HA/DR Recommendations
    • Deployment Recommendations
  • Presentation for Assessment Results