Koncepce testování CEPS HUB

Verze: 1.0 (návrh) · Stav: ke schválení Quality boardem Vlastník: QA Lead · Schvalovatelé: Head of Quality, CIO Vztah k dalším dokumentům:

• Test Policy definuje PROČ testujeme
• Master testovací strategie v2.2 definuje JAK testujeme (procesně)
• QA přístup (00-master) definuje quality gates napříč SDLC
• Tento dokument definuje JAK konkrétně testujeme — taxonomie testů, design, automation, perf/security/a11y/crypto, defekty, reporting, templates, CEPS-specifické scénáře
• Test Plan (per release) říká CO se v releasu testuje

0. TL;DR — co tento dokument přidává

Master strategie v2.2 a QA přístup definují procesní rámec. Tato koncepce vyplňuje metodologický obsah:

1. Taxonomie testů — 22 typů

1.1 Kompletní mapa typů

Poznámka k číslování: 40 položek, protože některé "typy" v tabulce 4.2 Master strategie jsou agregované rodiny (např. "Security" = SAST + SCA + DAST + Container + SBOM = 5 typů). Tato koncepce je rozbalí na konkrétní řemeslné jednotky.

1.2 Pyramida & vážení testů (cílový poměr)

                  ╱──────╲          Exploratory + UAT (5-8 %)
                 ╱  Manual ╲         business akceptace, charter sessions
                ╱───────────╲
               ╱   E2E auto  ╲       E2E (8-12 %)
              ╱  (Playwright) ╲      kritické business flows
             ╱─────────────────╲
            ╱  Integration + API ╲   Integration + Contract (20-25 %)
           ╱  (Pact, Schemathesis)╲  service-to-service, API contracts
          ╱───────────────────────╲
         ╱  Component + WireMock   ╲ Component (15-20 %)
        ╱   (in-process, with mocks)╲
       ╱─────────────────────────────╲
      ╱       Unit tests               ╲ Unit (40-55 %)
     ╱      (fast, isolated, JUnit)     ╲ business logic, validators
    ╱───────────────────────────────────╲

Plus transverzální vrstvy mimo pyramidu (běží paralelně):

• Performance (PREPROD, periodicky)
• Security (CI + PREPROD + ročně pentest)
• Accessibility (per UI release)
• Crypto/PKI (per release A + měsíčně)
• DR / Resilience (kvartálně + pololetně)
• Synthetic monitoring (PROD continuous)

1.3 Test rodiny vs. typy

2. Test design techniques

2.1 Black-box techniky (formal)

2.2 White-box techniky

2.3 Experience-based techniky

2.4 Risk-based prioritization

Risk Score = Impact × Probability   (z Master §2.2)

Pro každý epic / change vzniká:
1. Risk register (Jira issue type Risk)
2. Risk-weighted coverage matrix:
   - Risk 1-4   → smoke + spot check
   - Risk 5-9   → funkční + negativní
   - Risk 10-15 → + integration + perf baseline
   - Risk 16-25 → + DAST + pentest scope + DR drill consideration

2.5 Coverage typy a cíle

3. Test environments — kompletní specifikace

3.1 Environment matrix

3.2 Environment configuration baselines

Pro každé prostředí v Confluence "Environment Spec" (Master §5.4) doplnit:

SW komponenty (verzování):

• CEPS HUB version (Git tag)
• Camunda 8.5.x + ServiceTasks workers
• PostgreSQL 16.x + extensions (pgcrypto, pg_stat_statements)
• Kafka 3.6.x + Schema Registry + Connect
• Keycloak 24.x (OIDC, SAML)
• NGINX/Envoy reverse proxy
• Vault / KMS pro secrets
• Observability stack (Prometheus, Loki, OTel collector)

Network:

• Network topology diagram per env (AZ rozložení, redundance)
• Network policies (allowed inbound/outbound, NetworkPolicy YAML v Git)
• DNS, certifikáty (Let's Encrypt staging vs. produkční CA)
• Egress allowlist (ENTSO-E TP, NÚKIB portál — jen z PREPROD/PROD)

Test účty matrix:

• Per role (Žadatel, Schvalovatel, Dispečer, Admin, Auditor)
• IAM linkováno na Keycloak realm per env
• SoD (Segregation of Duties) test účty per kategorii rizika
• Service accounts pro mTLS integrace (rotace každých 90 dní)

Configuration drift detection:

• DEVOPS kontroluje týdně diff config vs. IaC Git repo
• Drift > 0 → ticket + investigation

3.3 Refresh policy + dependencies

3.4 Stop the environment — pravidla

QA může environmen "zastavit pro testování":

4. Test data management

4.1 Data taxonomie

4.2 Anonymizace pipeline (rozšíření Master §7.2)

PROD ─► snapshot (read-only) ─► anonymization env (izolované) ─►
   Tonic + custom rules ─► QA reviewer + DPO sign-off ─►
      anonymized dataset ─► encrypted at-rest ─► TEST/PREPROD

Anonymization rules per data type:

Audit:

• Anonymization script versioned v Git, code-reviewed
• DPO sign-off per export
• Audit log přístupů k anonymized data — 24 měsíců

4.3 Synthetic data factory pattern

Pro každou entitu (žádost, schválení, zpráva ENTSO-E) existuje factory v qa/data-factories repo:

# příklad pattern (pseudokód)
class VypadekFactory:
    def valid_default(self) -> Vypadek: ...
    def valid_with_kolize(self) -> Vypadek: ...
    def invalid_short_window(self) -> Vypadek: ...
    def boundary_max_duration(self) -> Vypadek: ...
    def boundary_min_duration(self) -> Vypadek: ...
    def historical_correction_30d(self) -> Vypadek: ...
    def out_of_order_timestamp(self) -> Vypadek: ...

Pravidla:

• Default = happy path
• Vždy mít metodu per boundary case
• Vždy mít metodu per negative case
• Seedable (deterministic reproducibility)
• Reference integrity zachována mezi faktory

4.4 Test data lifecycle

5. Test automation — framework a patterns

5.1 Automation strategy

Pro CEPS HUB platí 4 vrstvy automatizace:

Pravidlo: Pokud test má hodnotu jen v jednom releasu → manuální. Pokud má hodnotu opakovaně → automatizovat.

5.2 Framework choice

5.3 Patterns

Page Object Model (E2E):

// Příklad struktura
qa/e2e/
  ├── pages/
  │   ├── ZadostOdstavkyPage.ts
  │   ├── SchvalovatelDashboardPage.ts
  │   └── ...
  ├── components/    // shared UI components
  ├── flows/         // business flows (compose pages)
  │   ├── PodatZadostFlow.ts
  │   └── SchvalitZadostFlow.ts
  ├── tests/
  │   └── critical-journeys/
  └── fixtures/      // test data

API testing pattern (pytest):

# Příklad struktura
qa/api/
  ├── conftest.py        # session-scoped: client, auth, teardown
  ├── fixtures/
  │   └── factories.py   # data factories
  ├── tests/
  │   ├── zadosti/
  │   │   ├── test_post_zadost.py
  │   │   ├── test_get_zadost.py
  │   │   └── test_negative.py
  │   └── ...
  ├── contracts/         # Pact consumer tests
  └── utils/

Contract testing flow (Pact):

Consumer test ─► generates pact JSON ─► publish to Pact Broker ─►
   Provider verify (in CI) ─► broker stores verification result ─►
      can-i-deploy check before promote

Decision: WireMock vs Testcontainers

• WireMock — keď testujeme náš kód a externí service je stabilní (mock externí)
• Testcontainers — keď testujeme integraci s realnou závislostí (Postgres, Kafka)

5.4 Flaky test management

Definice flaky: Test, který v posledních 30 dnech selhal i prošel bez změny kódu nebo dat.

Workflow:

Flaky detected (ReportPortal trend) ─►
  Auto-tag "flaky" ─►
    Quarantine (vyřazen z PR gate, stále běží jako reportable) ─►
      Owner má 5 prac. dní na fix nebo deprecation ─►
        Po 5 dnech eskalace na QA Lead ─►
          Po 10 dnech delete + ticket "test gap"

Pravidla:

• Flaky rate > 2 % → block release (G6 gate)
• Žádný retry > 2× v CI
• Žádný "sleep" v testech — use waitFor/poll s timeout
• Test izolace: každý test má svoje data (factory + cleanup)
• Test idempotence: opakované spuštění dá stejný výsledek

5.5 Test code review

Test PR má stejně přísný review jako produkční kód:

• Test má jasný název (Given/When/Then nebo "should X when Y")
• Test je deterministický (žádné Date.now(), Random bez seed)
• Test je izolovaný (žádné sdílené stavy mezi testy)
• Test má jasné expected vs actual
• Test má cleanup (nebo @AfterEach)
• Test nečeká fixní čas (sleep(5000) zakázáno)
• Test pokrývá jeden konkrétní behavior (žádné "test všeho")
• Test má traceability (vazba na Xray nebo issue)

6. Performance testing methodology

6.1 Profily zátěže

6.2 NFR budget per endpoint

Pro každý kritický endpoint dokumentovaný v Performance Baseline (Master §16.4):

Regression pravidlo (Master §16.4 explicit): překročení baseline o > 10 % → formální Performance Regression Acceptance (varianta Risk Acceptance).

6.3 Performance test scenario template (k6)

Každý perf test má:

1. Hypothesis — co testujeme (např. "Endpoint POST /zadost zvládne 100 RPS s p95 ≤ 200 ms")
2. Setup — fixtures, auth tokens, warm-up
3. Stages — ramp-up, steady, ramp-down (definovaná zátěž)
4. Checks — per-request assertions
5. Thresholds — SLA limity (test fail při překročení)
6. Teardown — cleanup data
7. Reporting — HTML + JSON output to k6 cloud / Grafana

6.4 Performance gate

6.5 Capacity planning

Kvartálně (vlastník: SRE):

1. Load test s ramp-up nad cílovou zátěž
2. Měření capacity per komponenta
3. Update sizing model
4. Recommendation pro infra (scale-up nebo scale-out)

7. Security testing methodology

7.1 Defense-in-depth coverage

                    ┌──────────────────────────┐
                    │  External pentest         │  Ročně + při změně (A)
                    │  (manual, expert)         │
                    └──────────────────────────┘
                              │
                    ┌──────────────────────────┐
                    │  DAST (ZAP)               │  Per release (auto)
                    │  Runtime, black-box       │
                    └──────────────────────────┘
                              │
                    ┌──────────────────────────┐
                    │  IAST (volitelně)         │  Per release A (auto)
                    │  Instrumented runtime     │
                    └──────────────────────────┘
                              │
                    ┌──────────────────────────┐
                    │  SCA (Trivy, Snyk)        │  Per build (auto)
                    │  Dependencies, CVE        │
                    └──────────────────────────┘
                              │
                    ┌──────────────────────────┐
                    │  SAST (Semgrep, Sonar)    │  Per commit (auto)
                    │  Static code analysis     │
                    └──────────────────────────┘
                              │
                    ┌──────────────────────────┐
                    │  Secrets (gitleaks)       │  Per commit (auto)
                    │  No plaintext             │
                    └──────────────────────────┘
                              │
                    ┌──────────────────────────┐
                    │  Container scan (Trivy)   │  Per build (auto)
                    │  Image, OS packages       │
                    └──────────────────────────┘
                              │
                    ┌──────────────────────────┐
                    │  SBOM (Trivy/Syft)        │  Per build (auto)
                    │  Supply chain transparency│
                    └──────────────────────────┘

7.2 Threat modeling (per epic, kategorie A)

STRIDE kategorie pro každý nový komponent:

Output threat modelu:

• Threat list per komponent
• Mitigace per threat
• Test cases per mitigace (např. "Test 1.1: ne-auth user nemůže POST /zadost")

7.3 SAST configuration

Semgrep rulesets:

• OWASP Top 10 (default)
• CWE Top 25
• Custom ČEPS rules (např. "no plaintext key in code", "no SQL string concatenation")

SonarQube quality gate:

• 0 Critical, 0 High
• Code Smell density < 5 % (varování)
• Coverage gate (per kategorie)
• Duplicated code < 3 %

Per commit:

• Semgrep diff scan (jen change set)
• Gitleaks
• SonarQube full scan (mergeable size)

7.4 SCA + container scan

Trivy:

• Image scan při buildu (OS + packages + IaC + secrets)
• Severity threshold: 0 Critical, 0 High
• Mute povolen jen s expirací + důvodem v Trivy ignorefile (v Git)

Snyk:

• Application dependencies (Maven, npm, pip)
• License compliance (žádné GPL ve closed-source kódu)
• Auto-PR pro security updates

SBOM:

• CycloneDX format
• Uložen v artifact registry per build
• Diff scan vs. předchozí build (přidané závislosti = review)

7.5 DAST methodology (ZAP)

Per release v PREPROD:

1. Spin up auth context (test user per role)
2. ZAP baseline scan (passive) ─► all known endpoints
3. ZAP active scan (controlled) ─► known endpoints
   - SQL injection, XSS, CSRF, IDOR, SSRF, XXE
4. Validate findings (false positive mute s odůvodněním)
5. Generate report

Pravidla:

• 0 High/Critical findings blokuje release
• Medium findings — risk acceptance s expirací
• DAST scope zaregistrován v Confluence per modul

7.6 Pentest scope (ročně + při změně, kategorie A)

RoE (Rules of Engagement):

• Scope: kategorie A komponenty (např. dispečerské, ENTSO-E integrace, BPM)
• Out of scope: PROD bez explicitního souhlasu (ČEPS-internal pentester pouze)
• Time-boxing
• Pre-coordination s OPS (žádný DoS)
• Report retention: 5 let, klasifikace "Critical" v DMS

Output:

• Findings prioritized (Critical/High/Medium/Low)
• Remediation tracker (Jira issue type "Security Finding")
• Re-test po fix
• Lessons learned do test gap backlogu

8. Accessibility testing methodology (WCAG 2.1 AA)

8.1 Coverage matrix

8.2 axe-core integration (CI)

Per UI test run:
  Playwright + @axe-core/playwright ─►
    inject axe ─► run rules ─► assert no violations[Critical+Serious]

Rule config:

• Default WCAG 2.1 AA ruleset
• Custom rules: ČEPS branding (focus indicator viditelnost, contrast min)
• Mute povolen jen s odůvodněním v repo

8.3 Lighthouse CI per page (release A)

• Performance ≥ 90
• Accessibility ≥ 95
• Best Practices ≥ 90
• SEO neměřeno (interní app)

8.4 Manual checklist (per release, public UI)

• Keyboard navigation: all interactive elements reachable
• Focus visible: contrast meet WCAG
• Skip-to-content link funguje
• Screen reader (NVDA / VoiceOver): critical journey čtitelný
• Color contrast: text 4.5:1, large text 3:1
• No reliance on color alone (icons + text)
• Form labels association
• Error messages identifiable + actionable
• Images: alt text or aria-label
• Tables: headers, scope, caption
• Modals: focus trap + ESC
• Live regions for async content
• Language attribute set
• Page title unique per route

9. Crypto / PKI testing methodology

9.1 TLS testing (testssl.sh)

Per release + měsíčně:

# Příklad scope
testssl.sh --severity HIGH --jsonfile-pretty out.json https://api.ceps-hub.cz

Assertions:

• TLS 1.0, 1.1 — zakázáno
• TLS 1.2 — povoleno jen s strong ciphers
• TLS 1.3 — preferováno
• Heartbleed, POODLE, BEAST, CRIME — N/A
• HSTS — enabled, max-age ≥ 1 year
• Cert chain valid
• SSL Labs grade ≥ A (target A+)

9.2 Certificate lifecycle

9.3 mTLS testing (ENTSO-E ECP/EDX, Kafka)

Per release:

• Client cert validní (chain, expirace)
• Server akceptuje pouze trusted clients
• Revoked cert je odmítnut (OCSP/CRL)
• Cert rotation drill: rotate, validate connectivity restored < 5 min
• Mismatched cert (test) je odmítnut s clear error

9.4 HSM key rotation drill (ročně)

Scope:

• Keys: signing key (ENTSO-E messages), encryption key (at-rest)
• HSM: provider per ČEPS standard

Postup:

1. Pre-flight: backup existing key (export wrapped), document current key reference
2. Generate new key in HSM
3. Re-encrypt sample data with new key
4. Validate decrypt with new key
5. Update application config to use new key
6. Validate end-to-end (sample transaction)
7. Decommission old key (per regulation retention)

9.5 Encryption at-rest validation

Měsíčně:

• DB volume encryption status (pg_settings, cloud KMS check)
• Backup encryption (per cloud provider attestation)
• Restore test: backup → restore → validate data accessible after decryption

10. Integration & contract testing

10.1 Contract testing (Pact) — workflow

Consumer-driven contracts:

1. Consumer team píše test svého očekávání API
   pact.given('zadost-12345 exists').uponReceiving('GET /zadost/12345')
     .willRespondWith({status: 200, body: {...}})

2. Test vygeneruje pact.json

3. Publish to Pact Broker (per branch / per consumer version)

4. Provider team v CI verifikuje:
   pact-verifier --provider-base-url https://api... --pact-broker ...

5. Broker zaznamenává verification matrix:
   Consumer v1.2 × Provider v2.1 = PASS
   Consumer v1.2 × Provider v2.2 = FAIL

6. can-i-deploy check před deploy:
   pact-broker can-i-deploy --pacticipant Provider --version v2.2 --to-environment production

Pravidla:

• Žádný breaking change v API bez consumer notify (Pact verification fails)
• Provider versioning semver
• Consumer pact tests v CI per commit
• Provider verification per merge

10.2 Schema testing (Schemathesis)

Pro každý OpenAPI-described endpoint:

schemathesis run --checks all --hypothesis-database :memory: \
  https://api.../openapi.json

Generuje:

• Boundary value tests
• Schema validation
• Status code consistency
• Header validation
• Idempotence checks

10.3 Integration test patterns (Testcontainers)

@Testcontainers
class ZadostIntegrationTest {
  @Container static PostgreSQLContainer db = ...
  @Container static KafkaContainer kafka = ...
  @Container static GenericContainer keycloak = ...

  // Realistic deps, no global state, fast cleanup
}

Pravidla:

• Container start < 30s (pomocí small images, lazy init)
• Žádný shared state mezi testy
• Cleanup per @AfterEach
• Per-test isolation (random ports, fresh DB schema)

10.4 Service virtualization (WireMock)

Kdy WireMock místo Testcontainers:

• External service mimo náš tým (NÚKIB portal, banka API)
• External rate-limited service (ENTSO-E TP)
• Negative scenarios (5xx, timeout, malformed response)

Anti-patterns:

• WireMock pro vlastní službu (bez důvodu) — preferuj Testcontainers
• WireMock bez verifikace requestů (jen happy path)
• WireMock bez resetu mezi testy

11. E2E testing — critical journeys

11.1 Definice "Critical Journey"

Journey = business flow přes všechny systémy pro jednu user persona dosahující 1 business cíle.

CEPS HUB critical journeys (příklad):

11.2 E2E test scope rules

• 1 critical journey = 1 E2E test
• Test musí být deterministický (fixtures, no time-based race)
• Test musí být idempotentní (cleanup před + po)
• Test musí mít single responsibility (1 journey, ne 5)
• Test musí mít traceability (linknut na UC v Xray)
• Test časový limit: ≤ 5 min per test

11.3 E2E vs. lower-level test rozhodování

11.4 E2E infrastructure

• Playwright with parallel execution (4-8 workers)
• Browser pool: Chromium + Firefox + WebKit
• Trace + video + screenshot on failure
• Network HAR capture for debug
• Reporter: Allure (artefakty) + ReportPortal (trendy)

12. Exploratory testing — session-based

12.1 Charter pattern

Každá ET session má charter (5-W):

Charter ID:      ET-{release}-{n}
What:            Plánovaná odstávka — workflow schvalování
When:            T-2 dny před release
Where:           PREPROD env
Who:             1 QA, 2h session
Why:             Risk-based (kategorie A, recent changes in BPM)

Heuristics to apply:
  - SFDIPOT (Structure, Function, Data, Interface, Platform, Operations, Time)
  - Boundary values around schvalovací časové okno
  - Error guessing: timeout, expired session

Out of scope:
  - Performance (covered by load test)
  - Cross-browser (covered by E2E)

Test ideas (a priori):
  - Kolize 2 schvalovatelů
  - Storno schválené žádosti
  - Schválení po expiraci

Notes (during session):
  - [bug found] kolize → nedeterministic order
  - [question] co když schvalovatel revokovaný uprostřed?
  - [risk] nejasná error message při expiraci

Bugs filed: BUG-1234, BUG-1235
Time spent: 1h 45m
Coverage notes: 6 / 8 plánovaných oblastí

12.2 ET v cyklu releasu

12.3 Cíl ET sessions

• 10-20 % testovacího času pro kategorii A
• 5-10 % pro B
• Ad-hoc pro C

13. Smoke / Regression / UAT — standardy

13.1 Smoke test standard

Pravidla:

• Smoke je 100 % automatizovaný
• Smoke je 100 % deterministický (failed smoke = real problem)
• Smoke fail → auto-rollback (Master §15.3)

13.2 Regression suite standard

Regression suite = sada testů spuštěných před každým release do PREPROD.

Pravidla pro inclusion:

• Critical journey covered
• Stability ≥ 95 % (flaky rate < 5 %)
• Maintenance cost rozumný (automation effort < manual exec × 5)
• Pokrývá P0 nebo P1 issues z minulosti

Pravidla pro exclusion (move to archive nebo delete):

• Test neselhal 12+ měsíců a žádný change v dotčené oblasti → archive
• Test je flaky a fix > 1h effort → delete + replace
• Test pokrývá deprecated funkčnost → delete

Triáž v Test Closure (Master §11.4):

• Z TEST cyklu nové testy → review pro regression
• Stávající regression → review per release

13.3 UAT standard

UAT (User Acceptance Testing) workflow:

T-7 dní:  PO + QA sestaví UAT scope (business scenarios)
T-5 dní:  PREPROD env stabilizován, data refresh complete
T-3 dní:  UAT kickoff meeting (business + PO + QA observer)
T-1 dní:  UAT execution day(s)
T:        Go/No-Go meeting s UAT report

UAT pravidla:

• UAT scenarios = business acceptance criteria, ne technical tests
• UAT executor = real business user (ne QA)
• QA role = observer, support, evidence collection
• 100 % critical scenarios PASS pro Go
• Each finding → categorize (bug / feature gap / UX improvement)

13.4 UAT vs. funkční testy

14. DR / Failover / Chaos testing

14.1 DR drill cadence (Master §17.2)

14.2 Full DR drill scope (pololetně)

1. Trigger: scheduled, kategorie A scope
2. Notify: stakeholders (T-7 days)
3. Snapshot PROD baseline (RTO/RPO measurement reference)
4. Simulate disaster (controlled, izolovaný cluster):
   - Region/AZ outage
   - DB primary unavailable
   - Network partition
5. Execute DR runbook
6. Measure:
   - Actual RTO vs target (A: 1h, B: 4h, C: 24h)
   - Actual RPO vs target (A: 15 min, B: 1h, C: 24h)
   - Kafka/messaging resync lag
   - Transaction consistency
7. Post-drill:
   - Report (Master §17.3)
   - Action plan per finding
   - Updated runbook

14.3 Chaos / fault injection (ročně, kategorie A)

Hypothesis-driven experiments:

Hypothesis: System maintains < 1 % error rate when DB primary fails over
Steady state: Error rate < 0.1 %, p95 < 200 ms
Method: Kill DB primary container (controlled)
Blast radius: PREPROD only, isolated cluster
Stop conditions: Error rate > 5 %, p95 > 1 s
Observation: 10 min
Rollback: Restart primary or restore from snapshot

Závazná pravidla (Master §4.2):

• Pouze v izolovaném TEST/PREPROD
• Schválený scénář (Security + ARCH + OPS sign-off)
• Pre-flight: snapshot + rollback plan
• Run-time monitoring (live)
• Stop conditions definované
• Post-mortem povinný

14.4 Synthetic monitoring (PROD continuous)

15. Defect management — kompletní lifecycle

15.1 Defect lifecycle (rozšíření Master §10)

              ┌──────────────────────────────────────────────┐
              │                                              │
              ▼                                              │
        ┌─────────┐    ┌──────────┐    ┌──────────────┐    ┌─────────┐    ┌────────┐
        │ OPEN    │───►│IN PROGRESS│───►│FOR APPROVAL/  │───►│RESOLVED │───►│ CLOSED │
        │         │    │           │    │READY FOR RETEST│   │         │    │        │
        └─────────┘    └──────────┘    └──────────────┘    └─────────┘    └────────┘
              ▲              ▲                  │                ▲
              │              │  (retest fail)   │                │
              │              └──────────────────┘                │
              │                                                  │
              └──────────── (regress after close) ───────────────┘

Doplněné stavy / přechody (oproti Master §10.1):

• DEFERRED — defekt akceptovaný do dalšího releasu (waiver), z OPEN přes RACE
• DUPLICATE — z OPEN, link na master defekt
• CANNOT REPRODUCE — z IN PROGRESS po failed RCA, vrací do OPEN po nové evidence

15.2 Defect template (povinná pole)

ID:                BUG-{release}-{n}
Title:             [Module] — concise description
Severity:          S1 / S2 / S3 / S4 (technical impact)
Priority:          P1 / P2 / P3 / P4 (urgency)
Environment:       DEV / TEST / PREPROD / PROD
Found in:          Build / Release
Found by:          Tester / Automation / Production / Customer
Found via:         Manual / Automation / Synthetic / Incident

Steps to reproduce:
  1. ...
  2. ...
  3. ...

Expected result:
  ...

Actual result:
  ...

Evidence:
  - Screenshot:    [link]
  - Video:         [link]
  - HAR:           [link]
  - Logs (excerpt):[link]
  - Trace:         [link]

Affected components:
  - ...

Affected user roles:
  - ...

Workaround:
  - [None / Description]

Linked artifacts:
  - Test Execution: [Xray TE]
  - Test Case:      [Xray TC]
  - Story / Epic:   [Jira]
  - Risk:           [Jira RISK-X]
  - Fix version:    [Release]

Root cause (filled after fix):
  - Category:      [Code / Config / Data / Design / 3rd-party]
  - Detail:        ...
  - Prevention:    [Test gap / Code review / Design review / Monitoring]

15.3 Root Cause Analysis (RCA)

Povinné pro:

• P1 defekty v PROD (vždy)
• P1/P2 defekty escaped to PROD (post-release)
• Critical incidents

RCA workflow:

1. Immediate — stabilize (hotfix, rollback, mitigation)
2. Investigation — 5 Whys, timeline, evidence collection
3. Root cause — single sentence
4. Contributing factors — list
5. Corrective actions — short-term + long-term
6. Preventive actions — system changes, process changes, test gap
7. Lessons learned — into retro + knowledge base

Output: Post-Mortem document (do 5 prac. dní po incident).

15.4 Retest a regression check

Per fix:

15.5 Severity x Priority decision matrix (rozšíření §10.2)

15.6 Escalation triggers (rozšíření §10.4)

16. Test estimation a planning

16.1 Bottom-up estimation (per UC) — rozšíření Master §20

Formula:

Total effort =
  (Design + Data + Execute + Automate)
  × Integration_Factor (1.0-1.8)
  × Risk_Factor (1.0-1.5 dle kategorie)
  + Defect_Investigation (estimated 5-15 % effort)
  + Buffer (1.2-1.4 podle uncertainty)

16.2 Top-down estimation — T-shirt sizing (rozšíření §20.2)

16.3 Plán per release (rozšíření Master §1.2)

Test Plan obsahuje:

1. Scope (what's IN, what's OUT)
2. Test types per scope (function / NFR / compliance)
3. Risk register (per release)
4. Test environments + data plan
5. Schedule (per phase: design, prep, exec, regress)
6. Resource allocation (QA, automation, SRE)
7. Dependencies (external systems, data refresh)
8. Entry / Exit kritéria (per phase)
9. Acceptance criteria summary
10. Effort estimation + buffer
11. Risk assessment + mitigation
12. Reporting cadence
13. Go/No-Go criteria

16.4 Estimation evidence

Per release v Test Closure (Master §11.4):

• Estimated effort vs actual (variance)
• Defect rate estimated vs actual
• Lessons learned → tunning estimation model

17. Reporting & dashboards

17.1 Real-time dashboards (Grafana + ReportPortal)

Quality dashboard (Grafana, role-aware):

17.2 Test execution reports (ReportPortal + Allure)

Per test run:

• Pass/Fail per test
• Duration trend
• Flake history per test
• Failure category (network, data, code, infra)
• Linked artefakty (screenshots, video, trace, HAR, logs)

Per release:

• Cumulative pass rate
• Defect leakage (% found in TEST / PREPROD / PROD)
• Mean Time to Detect (MTTD) per defect
• Mean Time to Resolve (MTTR) per defect

17.3 Weekly QA report (rozšíření Master §13)

Povinné sekce:

1. Executive summary (3-5 vět) — kde stojí kvalita
2. Phase status — current phase + milestones
3. Coverage — UC, code, automation per modul
4. Defect trend — open/closed týden vs minulý, top 5 critical
5. Risk register changes — nová rizika, changes
6. Integration status — externí systémy (ENTSO-E TP, NÚKIB)
7. Environment health — stability, downtime, drift
8. Gate status — current phase entry/exit
9. Performance trend — perf baseline status
10. Security findings — new findings, remediation status
11. Go/No-Go recommendation — pre-release
12. Blockers + help needed

17.4 Release report (per release)

Obsah Release Evidence Pack (z QA přístupu, Příloha B) +:

• Comparison vs predcházející release (defect rate, coverage, perf)
• Lessons learned summary (z retro)
• Test gap backlog status

17.5 Audit reporty (per regulace)

NIS2 audit pack:

• Audit log test evidence
• DR drill reports (s RTO/RPO measurement)
• Incident reporting drill protocol
• SBOM per release

ZKB audit pack:

• Pentest reports (5 let)
• ISMS audit
• SoD test evidence
• Backup integrity log
• Crypto/TLS scan history

GDPR audit pack:

• Anonymization sign-offs (DPO)
• DSAR test execution
• Access audit log

ENTSO-E audit pack:

• TP sandbox test reports
• CIM compliance evidence

18. Test maintenance

18.1 Test refresh policy

18.2 Test debt management

Quarterly review (vlastník: QA Lead):

• Tests by age (oldest 10 %)
• Tests by flake rate (top 5 %)
• Tests by execution time (top 10 %)
• Tests by maintenance cost (per failure investigation hours)
• Decision: refactor / archive / delete

18.3 Test refactoring patterns

19. CI/CD integration

19.1 Pipeline stages (GitLab CI)

stages:
  - validate       # lint, format, dependency check
  - build          # compile, package
  - unit           # unit + component tests
  - sast-sca       # Semgrep + Trivy + Sonar + gitleaks
  - container      # image build + Trivy scan + Cosign sign + SBOM
  - integration    # Testcontainers, Pact verify
  - deploy-dev     # auto
  - smoke-dev      # post-deploy smoke
  - api-test       # Postman/pytest API suite
  - e2e-test       # Playwright critical journeys
  - deploy-test    # manual / scheduled
  - regression     # full regression suite
  - perf-baseline  # k6 baseline (PREPROD)
  - dast           # ZAP scan (PREPROD)
  - deploy-preprod # manual approval
  - uat-window     # manual (UAT execution)
  - deploy-prod    # manual approval (CIO/CAB)
  - smoke-prod     # post-deploy smoke
  - learning       # async: telemetry analysis, escaped defect detection

19.2 Parallel execution + sharding

• Unit tests: parallel by module
• API tests: parallel by suite
• E2E tests: sharded across 4-8 workers
• Performance: sequential (avoid noisy neighbor)

19.3 Test result aggregation

Per pipeline run:
  JUnit XML ─►
    ReportPortal (trend) +
    Xray (test execution) +
    Quality Studio (gate decision) +
    Allure (artefakty)

19.4 Selective test execution

Per MR:

• Affected modules detection (changed files → mapped to test suites)
• Run: unit + component for affected modules + smoke

Per main merge:

• Full unit + integration + smoke + contract

Per release branch:

• Full regression + perf baseline + security

20. CEPS-specific testing patterns

20.1 OASYS integration testing

Scope: Dispečerský systém ABB, OASYS export/import.

Test patterns:

• Datový formát kompatibility (OASYS spec)
• Synchronizace stavů (CEPS HUB ↔︎ OASYS)
• Reconciliation per den (diff detection)
• Failover during data sync

Test data:

• Anonymizovaný OASYS dataset (DPO sign-off)
• Synthetic boundary cases (max prvků, max odstávek)

20.2 ENTSO-E TP testing

Scope: Transparency Platform reporting (Commission Regulation EU 543/2013).

Test patterns:

• CIM data model compliance (XML schema validation)
• mRID uniqueness
• Time-series consistency (UTC, DST handling)
• Sandbox publication (ENTSO-E TP sandbox)
• Re-submission (correction within retention window)

Test data:

• Reference CIM datasets (versioned in Git)
• Sandbox account per env

20.3 BPM (Camunda 8.5) testing

Scope: Workflow engine pro schvalování.

Test patterns:

• Process instance creation
• Task assignment per role (RBAC)
• Decision tables (DMN) validation
• Boundary timers (escalation)
• Error events + compensation
• Migration of running instances on version change

Test framework: Camunda Test SDK + pytest

20.4 ECP/EDX testing

Scope: Energy Communication Platform + Data Exchange (mTLS).

Test patterns:

• mTLS handshake (client cert, server cert chain)
• Message signing + verification (ENTSO-E key)
• Retry / dead-letter handling
• Idempotence (duplicate message ID)
• Time-window validation

20.5 Kafka integration testing

Scope: Event streaming.

Test patterns:

• Producer-consumer contract
• Schema evolution (Schema Registry, backward compatibility)
• Partition assignment / rebalance
• Exactly-once semantics (idempotent producer, transactional consumer)
• Resync after failure
• Max lag thresholds

Tools: Testcontainers KafkaContainer + Schema Registry mock

20.6 PostgreSQL testing

Scope: Main DB.

Test patterns:

• Migration safety (Flyway/Liquibase forward + rollback)
• Index utilization (EXPLAIN ANALYZE in CI for new queries)
• Connection pool exhaustion
• Long-running query detection
• Backup integrity (pg_dump + restore + checksum)
• Point-in-time recovery (PITR) drill

20.7 Audit log testing (NIS2 vyžaduje)

Per release povinné:

• Login / logout logged
• Role change logged
• Data modification logged (who, when, what, before/after)
• Data access logged (PII access, sensitive read)
• Admin actions logged (config change, user mgmt)
• Failed authentication logged
• Audit log itself immutable (WORM)
• Audit log retention min. 3 roky
• Audit log export pro NÚKIB

21. TMMi L3 mapping

Cíl 2026 = TMMi Level 3 (Defined). Mapování process areas na tuto koncepci:

Cíl 2027 = TMMi Level 4 (Measured) v oblastech kategorie A:

22. Templates

22.1 Test Plan template

# Test Plan — {Release ID / Project}

## 1. Identifikace
- Release: ...
- Kategorie: A / B / C
- Vlastník Test Planu: QA Lead {jméno}
- Schvalovatel: PM + QA Lead

## 2. Scope
### V scope
- ...
### Out of scope
- ...

## 3. Testovací typy a rozsah
| Typ | Scope | Owner |
|---|---|---|
| Unit | ... | DEV |
| Integration | ... | DEV+QA |
| API | ... | QA |
| E2E | ... critical journeys | QA |
| Performance | ... (load + baseline) | SRE+QA |
| Security | SAST + SCA + DAST | SecOps |
| Accessibility | ... | QA+UX |
| ... | ... | ... |

## 4. Test Environments
- DEV: ...
- TEST: ...
- PREPROD: ... (UAT window: ...)

## 5. Test Data
- Synthetic: ...
- Anonymized: ... (DPO sign-off date)

## 6. Schedule
| Phase | Start | End | Milestone |
|---|---|---|---|
| Test Design | ... | ... | Design Gate |
| Test Prep | ... | ... | ... |
| Execution TEST | ... | ... | TEST Exit Gate |
| UAT | ... | ... | PREPROD Gate |
| Regression | ... | ... | ... |

## 7. Resource Plan
- QA: ... FTE × ... sprintů
- Automation: ...
- Externí: ...

## 8. Risk Register (top)
| Risk ID | Description | Score | Mitigation |
|---|---|---|---|
| ... | ... | ... | ... |

## 9. Entry / Exit Criteria
- Master §9.2 referenced + specifické:
  - ...

## 10. Acceptance Criteria
- ...

## 11. Reporting cadence
- Daily standup, Weekly QA Report, Go/No-Go meeting

## 12. Approval
- PO: ___
- PM: ___
- QA Lead: ___
- HoQ (kategorie A): ___

22.2 Test Case template (Xray-aligned)

# Test Case TC-{module}-{n}

## Identifikace
- ID: TC-{module}-{n}
- Title: ...
- Type: Functional / Negative / Boundary / Integration / E2E / etc.
- Priority: P0 / P1 / P2 / P3
- Risk weight: 1-25
- Automation status: Manual / Automated / Planned

## Linkování
- User Story: {Jira}
- AC: {AC #}
- Risk: {RISK-X}
- Requirement: {UC ID}

## Preconditions
- ...

## Test Data
- {dataset reference}

## Steps
| # | Action | Expected Result |
|---|---|---|
| 1 | ... | ... |
| 2 | ... | ... |

## Post-conditions
- ...

## Test Evidence (po execution)
- Test Execution ID: {Xray TE}
- Result: PASS / FAIL / BLOCKED / SKIPPED
- Defect: {BUG-X} (if FAIL)

22.3 Risk Register entry template

(Z Master §8.2 zachováno + doplněno o trigger.)

- ID: RISK-{release}-{n}
- Description: ...
- Impact (1-5): ...
- Probability (1-5): ...
- Risk Score: ...
- Owner: ...
- Mitigation plan: ...
- Residual Risk: ...
- Status: Open / Mitigated / Accepted / Closed
- Trigger for re-evaluation: ...
- Test coverage: {TC-X, TC-Y}

22.4 Exploratory Test Charter template

(Viz §12.1 výše.)

22.5 Test Report template (per release)

# Test Report — Release {ID}

## 1. Summary
- Test scope: ... (referenced Test Plan)
- Execution period: ... to ...
- Overall recommendation: GO / NO-GO / CONDITIONAL GO

## 2. Test Execution Summary
| Typ | Plánováno | Spuštěno | Pass | Fail | Blocked | Pass rate |
|---|---|---|---|---|---|---|
| Unit | ... | ... | ... | ... | ... | % |
| Integration | ... | ... | ... | ... | ... | % |
| API | ... | ... | ... | ... | ... | % |
| E2E | ... | ... | ... | ... | ... | % |
| Performance | ... | ... | ... | ... | ... | % |
| Security | ... | ... | ... | ... | ... | % |
| Accessibility | ... | ... | ... | ... | ... | % |
| UAT | ... | ... | ... | ... | ... | % |

## 3. Defect Summary
- Total found: ...
- By severity: S1=..., S2=..., S3=..., S4=...
- Open at release: P1=0, P2=0, P3=..., P4=...
- Escaped from previous: ...

## 4. NFR Results
- Performance: baseline +X %, all thresholds PASS / FAIL
- Security: 0 Critical, 0 High; medium N findings (acceptance attached)
- Accessibility: WCAG 2.1 AA PASS / FAIL per page
- Crypto/PKI (A): PASS / FAIL

## 5. Risk Status
- Top 5 active risks, residual scores
- Risk acceptance count

## 6. Coverage
- UC coverage: %
- Code coverage: %
- Risk-weighted coverage: %
- Automation coverage: %

## 7. Compliance
- NIS2: audit log test PASS, DR drill {date}
- ZKB: SoD test PASS, pentest report (link)
- GDPR: DSAR test PASS, anonymization sign-off (link)
- ENTSO-E: TP sandbox PASS

## 8. Issues / Blockers
- ...

## 9. Lessons Learned (preliminary)
- ...

## 10. Approval
- QA Lead: ___
- PM: ___
- HoQ (kategorie A): ___

23. Implementace této koncepce

23.1 Co je hotové (existující)

• Master strategie v2.2 (process + organization)
• Schválený toolchain (Master §6.1)
• Xray test management
• Kategorizace A/B/C
• DoR + DoD (Master §11)
• Gate workflow CI/CD (Master §9)
• Risk management (Master §8)
• Compliance mapping (Master §14)

23.2 Co tento dokument přidává

• Plná taxonomie 40 typů testů
• Test design techniques katalog
• Environment + data deep dive
• Performance / Security / Accessibility / Crypto metodologie
• Contract & Integration patterns
• E2E + Exploratory koncepce
• Defect lifecycle + RCA
• Templates per artefakt
• CEPS-specifické testing patterns
• TMMi L3 → L4 mapping

23.3 Co je třeba doimplementovat

Konec dokumentu.