HTL-09 – TEST PLAN & VALIDATION MATRIX (Outline v0.1)

• HTL-09 – TEST PLAN & VALIDATION MATRIX (Outline v0.1)
• 1. Purpose
  • 1.1 Document Objective
  • 1.2 Authority
  • 1.3 Acceptance Authority
• 2. Scope
  • 2.1 In-Scope
  • 2.2 Out-of-Scope
• 3. Definitions
  • 3.1 Unit Test
  • 3.2 Integration Test
  • 3.3 Stress Test
  • 3.4 Soak Test
  • 3.5 Failure Injection
  • 3.6 Acceptance Test
  • 3.7 Test Environment
  • 3.8 Pass/Fail Criteria
• 4. Assumptions
  • 4.1 Baseline Deployment
  • 4.2 Test Environment
  • 4.3 Tools Assumption
• 5. Test Architecture Overview
  • 5.1 Test Topology Diagram
  • 5.2 Test Data Flow Diagram
• 6. Technical Specification – Test Categories
  • 6.1 Unit Testing
  • 6.2 Integration Testing
  • 6.3 Electrical Validation
  • 6.4 Failure Injection Testing
  • 6.5 Stress Testing
  • 6.6 Soak Testing
  • 6.7 OTA Validation
  • 6.8 Security Validation
  • 6.9 Commissioning Validation
• 7. Constraints
  • 7.1 Hardware Availability Constraint
  • 7.2 Test Equipment Limitation
  • 7.3 Time Constraint
  • 7.4 Field Access Constraint
  • 7.5 Budget Constraint
• 8. Failure Handling During Testing
  • 8.1 Test Abort Criteria
  • 8.2 Escalation Rule
  • 8.3 Log Capture Requirement
  • 8.4 Retest Policy
  • 8.5 Defect Classification
• 9. Validation Matrix
  • Sample Validation Matrix (Baseline)
• 10. Open Issues
• 11. Revision History

1. Purpose

1.1 Document Objective

HTL-09 menjadi:

• Referensi formal QA
• Gate sebelum produksi
• Checklist commissioning
• Validasi failure containment
• Validasi degradasi sistem

Tidak ada site yang boleh dinyatakan Production Ready tanpa lulus HTL-09.

1.2 Authority

HTL-09 mengikat:

• QA Team
• Node Team
• Gateway Team
• Server Team
• Electrical Team
• Field Team

Semua defect harus dicatat sebelum sign-off.

1.3 Acceptance Authority

Production Ready hanya dapat dinyatakan oleh:

• QA Lead (validasi teknis)
• Electrical Lead (validasi panel & safety)
• System Architect (validasi arsitektur menyeluruh)

Tanpa 3 approval tersebut → site tidak boleh aktif produksi.

2. Scope

2.1 In-Scope

Kategori pengujian:

• Unit Test
• Integration Test
• Electrical Validation
• Failure Injection Test
• Stress Test
• Soak Test
• OTA Validation
• Security Validation
• Commissioning Validation

Semua domain HTL-00 s/d HTL-08 harus tervalidasi.

2.2 Out-of-Scope

Tidak termasuk:

• Cloud scalability
• Multi-site federation
• Studi agronomi jangka panjang

3. Definitions

3.1 Unit Test

Pengujian komponen terisolasi tanpa dependensi layer lain.

3.2 Integration Test

Pengujian end-to-end antar layer (Node → Gateway → Server).

3.3 Stress Test

Pengujian beban maksimum sistem dalam waktu singkat.

3.4 Soak Test

Pengujian jangka panjang (48–72 jam) untuk stabilitas.

3.5 Failure Injection

Simulasi kegagalan untuk menguji containment dan recovery.

3.6 Acceptance Test

Pengujian final sebelum site dinyatakan produksi.

3.7 Test Environment

Lingkungan pengujian:

• Lab mode
• Field mode
• Simulasi relay chain

3.8 Pass/Fail Criteria

Setiap test harus memiliki:

• Expected behavior
• Measured behavior
• Numeric threshold
• Clear PASS/FAIL

4. Assumptions

4.1 Baseline Deployment

Per-site:

• 1 Gateway
• 10–15 Node
• 1 Raspberry Pi

4.2 Test Environment

Dua mode:

1. Lab mode

   • Simulated relay chain
   • Controlled power

2. Field mode

   • Real wiring
   • Real actuator load

4.3 Tools Assumption

Minimal alat:

• MQTT monitoring tool
• Serial log capture
• Power analyzer
• Multimeter
• Network analyzer
• Log export tool

Tanpa alat ini, validasi tidak sah.

5. Test Architecture Overview

Tujuan section ini adalah memastikan pengujian dilakukan pada topologi yang merepresentasikan kondisi produksi sebenarnya.

Pengujian tidak boleh dilakukan pada konfigurasi yang tidak mencerminkan baseline deployment.

5.1 Test Topology Diagram

✔ Komponen Wajib dalam Test Setup

1. Node Cluster (10–15 unit)

   • Minimal 3 node untuk relay-chain test
   • Minimal 1 node dengan actuator real

2. Relay Chain Simulation

   • Parent-child routing aktif
   • Hop limit aktif

3. Gateway

   • ESP-NOW coordinator aktif
   • MQTT bridge aktif

4. Raspberry Pi

   • MQTT broker
   • DB
   • Dashboard

5. Test Client

   • MQTT monitor
   • Log collector
   • Command generator

6. Power Disturbance Injector (simulasi)

   • Manual power cut
   • Brownout simulation
   • Surge simulation (terkontrol)

✔ Test Mode

Mode A – Lab Mode

• Beban dummy (resistive load)
• Relay switching tanpa pompa real

Mode B – Field Mode

• Pompa real
• Panel real
• Grounding real

Semua release firmware harus lulus Lab Mode sebelum Field Mode.

5.2 Test Data Flow Diagram

✔ Jalur Data yang Harus Diuji

1. Telemetry Path Node → Relay → Gateway → MQTT → DB

2. Command Path HMI → Server → MQTT → Gateway → Node

3. ACK Path Node → Gateway → MQTT → Server → HMI

4. Config Path Server → MQTT → Gateway → Node

5. OTA Path Server → Metadata → Node → Hash Verify → Apply

6. Failure Injection Path Injected Fault → Detection → Containment → Recovery

✔ Validasi Flow

Setiap jalur harus:

• Diverifikasi latency
• Diverifikasi reliability
• Diverifikasi idempotency
• Diverifikasi TTL enforcement

6. Technical Specification – Test Categories

6.1 Unit Testing

✔ 6.1.1 Node Unit Tests

Semua test harus dilakukan dengan log capture aktif.

✔ 6.1.2 Gateway Unit Tests

✔ 6.1.3 Server Unit Tests

6.2 Integration Testing

Semua integration test harus dilakukan dengan minimal 10 node aktif.

6.3 Electrical Validation

Electrical validation wajib dilakukan di Field Mode.

6.4 Failure Injection Testing

Setiap skenario wajib memiliki:

• Expected behavior
• Measured behavior
• Pass/Fail criteria

6.5 Stress Testing

Stress test dilakukan minimal 30 menit per skenario.

6.6 Soak Testing

Durasi: 48–72 jam continuous.

Monitoring:

• Memory usage
• MQTT reconnect count
• DB write stability
• Watchdog event

Pass criteria:

• No memory leak
• No unexpected reset
• No routing collapse
• Stable CPU usage

6.7 OTA Validation

6.8 Security Validation

6.9 Commissioning Validation

Checklist wajib sebelum go-live:

• Wiring inspection OK
• Grounding verified
• Node pairing valid
• Gateway online
• Pi services healthy
• Initial config loaded
• Alarm test pass
• Manual override test pass
• Brownout simulation pass

Tanpa checklist lengkap → site tidak boleh aktif.

7. Constraints

7.1 Hardware Availability Constraint

• Minimal 10 Node untuk integration & stress test
• Minimal 1 panel real untuk electrical validation
• Spare relay & actuator wajib tersedia untuk uji destruktif ringan

Tanpa hardware representatif → hasil test tidak valid.

7.2 Test Equipment Limitation

Jika alat berikut tidak tersedia:

• Power analyzer
• Network analyzer
• Log capture tool

Maka:

• Electrical validation terbatas
• Stress & latency measurement tidak presisi
• QA harus mencatat keterbatasan

7.3 Time Constraint

Minimum testing window:

• Unit test: per commit
• Integration test: per release candidate
• Soak test: 48–72 jam
• Electrical test: minimal 1 hari penuh

Testing dipercepat → risiko produksi meningkat.

7.4 Field Access Constraint

Jika field access terbatas:

• Electrical validation harus dijadwalkan
• Surge simulation hanya dilakukan di lab
• Field commissioning tidak boleh dilewati

7.5 Budget Constraint

Jika keterbatasan anggaran:

• Prioritaskan failure injection & soak test
• Electrical validation tidak boleh dihilangkan
• Security validation minimal tetap wajib

8. Failure Handling During Testing

8.1 Test Abort Criteria

Test harus dihentikan jika:

• Hardware rusak permanen
• Electrical unsafe condition
• Panel overheat
• Surge simulation di luar batas aman

Keselamatan lebih tinggi dari validasi.

8.2 Escalation Rule

Level 1 – QA Engineer

• Catat defect
• Ulang test

Level 2 – Component Owner

• Debug & fix
• Release patch

Level 3 – Architect

• Review desain
• Update HTL document jika perlu

8.3 Log Capture Requirement

Setiap test wajib memiliki:

• Timestamp
• Firmware version
• Hardware version
• Config version
• Log file attachment

Tanpa log → test dianggap tidak valid.

8.4 Retest Policy

Jika FAIL:

1. Fix implemented
2. Unit test ulang
3. Integration test ulang
4. Soak test ulang (jika defect kritikal)

No bypass allowed.

8.5 Defect Classification

Severity:

• Critical → Unsafe actuator behavior
• Major → Loss of control
• Minor → UI/Logging issue
• Cosmetic → Tidak mempengaruhi fungsi

Critical & Major wajib ditutup sebelum production.

9. Validation Matrix

Format wajib:

| Test ID | Category | Scenario | Expected Result | Pass/Fail | Owner |

Sample Validation Matrix (Baseline)

Matrix lengkap harus diisi sebelum go-live.

10. Open Issues

Harus dikunci sebelum production freeze:

1. Minimum soak duration final?
2. UPS mandatory selama test?
3. Field trial duration minimum?
4. Spare hardware policy?
5. Automated CI pipeline untuk firmware?
6. Regression test baseline per release?
7. Numeric latency threshold final?

Tanpa angka final → acceptance ambigu.

11. Revision History