Data life cycle basics

Data is not a static pile of files. In any serious system it is born, shaped, stored, moved, interpreted, retained, and eventually destroyed—or it lingers forever and becomes a liability. The data life cycle is the standard way computer science and information systems curricula name that path.

This note is a breakdown of the life cycle as a knowledge area: definitions, stages, cross-cutting concerns, and how the idea connects to databases, operating systems, security, and analytics.

One-sentence crystal

The data life cycle is the ordered story of information from intent and creation, through collection, processing, storage, use, and sharing, to long-term retention or disposal—with quality, security, privacy, and provenance binding every stage.

---

1. Why this is a CS knowledge area

Computer science is not only algorithms and machines. A large fraction of real systems is moving bits that mean something under constraints of correctness, cost, latency, law, and trust.

If you can only “query a table” but cannot say where data came from, who may see it, how long it lives, and how it dies, the knowledge area is incomplete.

Learning outcomes

You should be able to:

1. Name the main stages of a data life cycle and what happens in each.
2. Separate data, metadata, and derived data.
3. Point to cross-cutting concerns (quality, security, privacy, provenance) at any stage.
4. Map a familiar system (app, lab logger, firmware telemetry) onto the cycle.
5. Ask stage-appropriate questions in design or review.

---

2. Bird’s-eye cycle

Stages are often drawn as a ring because disposal can free capacity for new collection, and because “use” often feeds new creation (feedback). Linearity is still useful for teaching.

                    ┌──────────────┐
           ┌───────►│ 1. Plan /    │
           │        │    create    │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 2. Collect / │
           │        │    ingest    │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 3. Process / │
           │        │  transform   │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 4. Store /   │
           │        │   persist    │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 5. Share /   │
           │        │  use / analyze│
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 6. Archive / │
           │        │   retain     │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           └────────│ 7. Dispose / │
                    │   destroy    │
                    └──────────────┘

   Cross-cutting (every stage): quality · security · privacy · provenance · metadata

Some curricula insert “publish / distribute” as its own stage, or fold archive into storage. The seven stages below keep decisions visible.

---

3. Stage-by-stage breakdown

3.1 Plan and create

Before the first row exists, someone decides what the world is allowed to mean as data.

Create includes synthetic generation, form design, sensor selection, and log schema design. Bad planning cannot be fixed by a prettier dashboard later.

3.2 Collect and ingest

Data enters the system: sensors, user input, APIs, files, scrapers, message queues, lab instruments.

Ingest is the first place loss, duplication, and clock skew show up. Time stamps and source IDs belong here, not as afterthoughts.

3.3 Process and transform

Raw inputs become usable representations: clean, join, normalize, encode, aggregate, feature-engineer, encrypt, anonymize.

  raw event  →  validate  →  enrich  →  transform  →  curated record

Rule of honesty: every transform can destroy information or inject bias. Document it; version it; test it.

3.4 Store and persist

Persistence is where data lives so later stages can find it: filesystems, object stores, databases, data lakes/warehouses, embedded flash logs, cold tape.

Storage is not neutral. Choosing a log file vs a transactional DB is a life-cycle decision, not only a performance tweak.

3.5 Share, use, and analyze

The point of most data: decisions, products, models, human reading.

Share implies authorization, contracts, and often rate limits. “Everyone can SELECT *” is not a share strategy; it is an incident waiting for a calendar date.

3.6 Archive and retain

Not all data stays hot. Archive moves infrequently used data to cheaper, slower media with explicit retention policy.

Archive is still in the life cycle: forgotten archives are silent risk (breach surface, wrong historical truth).

3.7 Dispose and destroy

End of life: delete, shred, crypto-erase, or physically destroy media—according to sensitivity.

Disposal is a first-class stage. Curriculum that stops at “store and analyze” trains incomplete engineers.

---

4. Cross-cutting concerns (bind every stage)

These are not “extra chapters after the cycle.” They are properties of the cycle.

4.1 Data quality

Quality is use-relative: lab telemetry “good enough for debug” may be unfit for safety claims.

4.2 Security

Confidentiality, integrity, availability—and accountability—applied to data at rest, in transit, and in use (access control, encryption, logging, backup security).

4.3 Privacy

Not identical to security. Privacy asks whether collection and use respect people and law (minimization, purpose limitation, consent, subject rights). A well-encrypted hoard of unnecessary personal data can still be a privacy failure.

4.4 Provenance and lineage

Where did this value come from? Source system, transform version, time, operator. Lineage enables debug, audit, and scientific reproducibility. Without it, every dashboard is a rumor.

4.5 Metadata

Data about data: schemas, owners, sensitivity labels, retention tags, dictionary definitions. Metadata is what makes stages operable by people who did not create the data.

  data value  +  metadata  +  provenance  ≈  something you can trust or reject consciously

---

5. Data vs information vs knowledge (vocabulary)

Curriculum often layers meanings:

The life cycle primarily moves data and metadata; information appears when use attaches purpose; knowledge often lives in models, docs, and trained people—and can feed new data (labels, features, decisions logged as events).

---

6. One example mapped end-to-end

Scenario: temperature samples from a device for a dashboard and monthly report.

Cross-cutting: range checks (quality), TLS and auth (security), no personal location unless needed (privacy), schema version in payload (provenance).

---

7. Design and review checklist

1. Purpose — Why does this data exist?
2. Birth — Who/what creates it; under what consent or right?
3. Shape — Schema, units, time, identifiers, versions.
4. Path — How does it move (protocols, batches, owners)?
5. Truth — How is quality measured and repaired?
6. Access — Who may read/write/share; how is that enforced?
7. Copies — Backups, replicas, exports—where are they?
8. Clock — Retention and legal hold.
9. Death — How is destruction defined and proven?
10. Feedback — Does use create new data that re-enters the cycle?

---

Closing

The data life cycle is a curriculum backbone for treating information as something that is planned, born, refined, kept, used, and ended—not as a spreadsheet that “just is.”

Master the stages, the cross-cutting duties (quality, security, privacy, provenance, metadata), and the habit of mapping any real system onto the cycle. Algorithms and databases then have a place to live; without the cycle, they are tools without a biography of the data they touch.