Compare commits

...

3 Commits

8 changed files with 147 additions and 90 deletions

View File

@@ -8,7 +8,7 @@ application state.
## Variables
```yaml
narratio_version: v2.2.0
narratio_version: v3.0.0
narratio_repository_url: https://gitea.maximumdirect.net/eric/narratio.git
narratio_source_dir: /usr/local/src/narratio
narratio_binary_path: /usr/local/bin/narratio
@@ -64,19 +64,14 @@ narratio_seriatim_env_backchannel_max_duration: null
narratio_seriatim_env_filler_max_duration: null
narratio_audita_binary: null
narratio_audita_timeout: null
narratio_audita_llm_api_key: null
narratio_audita_credentials: null
narratio_audita_modules: null
narratio_audita_base_url: null
narratio_audita_model: null
narratio_audita_total_llm_concurrency: null
narratio_audita_proposal_llm_concurrency: null
narratio_audita_validation_model: null
narratio_audita_validation_llm_concurrency: null
narratio_audita_overlay: null
narratio_audita_proposal_profile: null
narratio_audita_validation_profile: null
narratio_audita_transcript_description: null
narratio_audita_config_path: null
narratio_audita_output_schema: null
narratio_audita_work_dir_retention: null
narratio_audita_report: null
narratio_normalize_output_path: null
narratio_normalize_output_schema: null
narratio_normalize_report: null
@@ -105,11 +100,11 @@ narratio_scriptorium_artifact_families: null
narratio_notarius_enabled: null
narratio_notarius_binary: null
narratio_notarius_config_path: null
narratio_notarius_config_profile: null
narratio_notarius_config_overlay: null
narratio_notarius_pipeline_id: null
narratio_notarius_timeout: null
narratio_notarius_working_directory: null
narratio_notarius_llm_api_key: null
narratio_notarius_credentials: null
narratio_notarius_resume: null
narratio_notarius_chunk_cache: null
narratio_notarius_references: null
@@ -136,18 +131,75 @@ falsey values are preserved. `narratio_publish_outputs`,
`narratio_notarius_references`, `narratio_notarius_outputs`,
`narratio_scriptorium_artifacts`, and `narratio_scriptorium_artifact_families`
are complete native caller values, so callers may compose them before assigning
them and the role does not merge or rewrite them. Credential fields name
environment variables only; credential values remain outside this role and are
the caller's responsibility. `narratio_notarius_llm_api_key` names the
environment variable that Narratio forwards to Notarius. The optional
`narratio_notarius_config_profile` selects a Notarius ConfigKit overlay without
coupling it to Narratio's own profile name. The optional `narratio_notarius_resume`
them and the role does not merge or rewrite them. The optional
`narratio_notarius_config_overlay` selects a Notarius ConfigKit overlay without
coupling it to Narratio's own overlay name. The optional `narratio_notarius_resume`
and `narratio_notarius_chunk_cache` settings expose Narratio's native extraction
and chunk-plan choices; this generic role assigns no D&D-specific credential or
cache policy.
ConfigKit imports and profile overlays are externally managed, and this role
ConfigKit imports and selected overlays are externally managed, and this role
does not install the downstream applications, campaign content, or suite policy.
## Upgrading from Narratio v2
Narratio v3 removes these role variables and their native fields:
`narratio_audita_llm_api_key`, `narratio_audita_base_url`,
`narratio_audita_model`, `narratio_audita_total_llm_concurrency`,
`narratio_audita_proposal_llm_concurrency`,
`narratio_audita_validation_model`,
`narratio_audita_validation_llm_concurrency`,
`narratio_audita_work_dir_retention`, and `narratio_audita_report`.
Replace `narratio_notarius_llm_api_key` with the list-valued
`narratio_notarius_credentials`; rename `narratio_notarius_config_profile` to
`narratio_notarius_config_overlay`. Existing deployments that cannot migrate
these inputs should retain their previous role pin. See the tagged
[v3.0.0 release notes](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/releases/v3.0.0.md).
`narratio_audita_credentials` and `narratio_notarius_credentials` are lists of
environment-variable or secret-reference **names**, never credential values.
Callers supply nonempty values through Narratio's run environment or supported
secret source. Narratio resolves configured names when loading a pipeline,
including when the downstream stage will be skipped. Audita v2 needs names for
every credential in its configured backends, even backends not selected by the
current profiles; Notarius names must match its selected provider profiles.
The role does not inspect downstream files or resolve secrets. Null omits a
list or selector, `[]` emits an explicit empty list, and `""` emits an explicit
empty selector. A Narratio overlay replaces a credential list rather than
appending to it.
Audita v2 owns its configuration, backend/model definitions, PromptKit profiles,
reporting, and debug policy. Configure it independently before using polish;
the role does not generate downstream files or guess replacements for the
removed tuning fields. The Audita overlay, proposal profile, and validation
profile selectors are independent of each other and Narratio's selected
overlay. The Notarius `config_overlay` selector is likewise independent; null
adds no downstream selection, while empty, `default`, and named strings are
passed to Notarius unchanged. Selected downstream features require compatible
Audita and Notarius executables; this role adds no version gate. See the tagged
[Audita](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/integrations/audita.md)
and [Notarius](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/integrations/notarius.md)
integration contracts.
For Narratio itself, replace `--profile` with `--overlay` and move selected
`profiles/<name>.yml` files to `overlays/<name>.yml` beside the pipeline root.
Explicit ConfigKit imports from a directory named `profiles` remain valid;
PromptKit LLM-profile files are a separate concept. Notarius's renamed
`config_overlay` selects its own `overlays/` files, replacing its retired
configuration-profile selection. The D&D suite's testing file and selector
migrate later with coordinated suite integration, not in this generic role.
Before opening an affected existing session, follow the tagged
[manifest migration procedure](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/operations.md#migrating-manifest-overlay-provenance):
back up the local manifest and change only
`effective_config.selected_profile` to `selected_overlay` (or remove the
retired informational member), preserving local progress and checkpoints.
Immutable remote snapshots are not edited in place. For named Notarius
selectors, the renamed field changes extraction resume evidence: the next
applicable run regenerates extraction with normal dependent invalidation.
Omitted/base selection does not gain invalidation solely from the rename.
The role performs no manifest migration, cache cleanup, or blanket forced
reruns.
## Example
```yaml
@@ -166,10 +218,11 @@ pipeline through Narratio as its application account and requires immediate
clean repeated convergence. It explicitly selects `v2.1.0` and checks native
version output, then combines missing-binary recovery with a caller-supplied Go
executable while dependency management is disabled. A representative pipeline
checks null omission, falsey values, caller-composed native collections, and
the native Notarius credential-reference, `resume`, and chunk-cache fields
before native validation and a clean repeat. The suite restores the baseline
deployment during cleanup.
checks null omission, falsey values, caller-composed native collections,
credential-name lists, downstream selectors, and the native Notarius `resume`
and chunk-cache fields. It validates named and explicit-empty configurations
offline and requires clean repeats. The suite restores the baseline deployment
during cleanup.
```sh
tests/integration

View File

@@ -1,5 +1,5 @@
---
narratio_version: v2.2.0
narratio_version: v3.0.0
narratio_repository_url: https://gitea.maximumdirect.net/eric/narratio.git
narratio_source_dir: /usr/local/src/narratio
narratio_binary_path: /usr/local/bin/narratio
@@ -49,19 +49,14 @@ narratio_seriatim_env_backchannel_max_duration: null
narratio_seriatim_env_filler_max_duration: null
narratio_audita_binary: null
narratio_audita_timeout: null
narratio_audita_llm_api_key: null
narratio_audita_credentials: null
narratio_audita_modules: null
narratio_audita_base_url: null
narratio_audita_model: null
narratio_audita_total_llm_concurrency: null
narratio_audita_proposal_llm_concurrency: null
narratio_audita_validation_model: null
narratio_audita_validation_llm_concurrency: null
narratio_audita_overlay: null
narratio_audita_proposal_profile: null
narratio_audita_validation_profile: null
narratio_audita_transcript_description: null
narratio_audita_config_path: null
narratio_audita_output_schema: null
narratio_audita_work_dir_retention: null
narratio_audita_report: null
narratio_normalize_output_path: null
narratio_normalize_output_schema: null
narratio_normalize_report: null
@@ -90,11 +85,11 @@ narratio_scriptorium_artifact_families: null
narratio_notarius_enabled: null
narratio_notarius_binary: null
narratio_notarius_config_path: null
narratio_notarius_config_profile: null
narratio_notarius_config_overlay: null
narratio_notarius_pipeline_id: null
narratio_notarius_timeout: null
narratio_notarius_working_directory: null
narratio_notarius_llm_api_key: null
narratio_notarius_credentials: null
narratio_notarius_resume: null
narratio_notarius_chunk_cache: null
narratio_notarius_references: null

View File

@@ -54,19 +54,14 @@ argument_specs:
narratio_seriatim_env_filler_max_duration: {type: float, required: false}
narratio_audita_binary: {type: str, required: false}
narratio_audita_timeout: {type: str, required: false}
narratio_audita_llm_api_key: {type: str, required: false}
narratio_audita_credentials: {type: list, elements: str, required: false}
narratio_audita_modules: {type: list, required: false}
narratio_audita_base_url: {type: str, required: false}
narratio_audita_model: {type: str, required: false}
narratio_audita_total_llm_concurrency: {type: int, required: false}
narratio_audita_proposal_llm_concurrency: {type: int, required: false}
narratio_audita_validation_model: {type: str, required: false}
narratio_audita_validation_llm_concurrency: {type: int, required: false}
narratio_audita_overlay: {type: str, required: false}
narratio_audita_proposal_profile: {type: str, required: false}
narratio_audita_validation_profile: {type: str, required: false}
narratio_audita_transcript_description: {type: str, required: false}
narratio_audita_config_path: {type: str, required: false}
narratio_audita_output_schema: {type: str, required: false}
narratio_audita_work_dir_retention: {type: str, required: false}
narratio_audita_report: {type: bool, required: false}
narratio_normalize_output_path: {type: str, required: false}
narratio_normalize_output_schema: {type: str, required: false}
narratio_normalize_report: {type: bool, required: false}
@@ -95,11 +90,11 @@ argument_specs:
narratio_notarius_enabled: {type: bool, required: false}
narratio_notarius_binary: {type: str, required: false}
narratio_notarius_config_path: {type: str, required: false}
narratio_notarius_config_profile: {type: str, required: false}
narratio_notarius_config_overlay: {type: str, required: false}
narratio_notarius_pipeline_id: {type: str, required: false}
narratio_notarius_timeout: {type: str, required: false}
narratio_notarius_working_directory: {type: str, required: false}
narratio_notarius_llm_api_key: {type: str, required: false}
narratio_notarius_credentials: {type: list, elements: str, required: false}
narratio_notarius_resume: {type: bool, required: false}
narratio_notarius_chunk_cache: {type: str, required: false}
narratio_notarius_references: {type: dict, required: false}

View File

@@ -114,7 +114,7 @@ seriatim:
{% endif %}
{% endif %}
{% endif %}
{% if narratio_audita_binary is not none or narratio_audita_timeout is not none or narratio_audita_llm_api_key is not none or narratio_audita_modules is not none or narratio_audita_base_url is not none or narratio_audita_model is not none or narratio_audita_total_llm_concurrency is not none or narratio_audita_proposal_llm_concurrency is not none or narratio_audita_validation_model is not none or narratio_audita_validation_llm_concurrency is not none or narratio_audita_transcript_description is not none or narratio_audita_config_path is not none or narratio_audita_output_schema is not none or narratio_audita_work_dir_retention is not none or narratio_audita_report is not none %}
{% if narratio_audita_binary is not none or narratio_audita_timeout is not none or narratio_audita_credentials is not none or narratio_audita_modules is not none or narratio_audita_overlay is not none or narratio_audita_proposal_profile is not none or narratio_audita_validation_profile is not none or narratio_audita_transcript_description is not none or narratio_audita_config_path is not none or narratio_audita_output_schema is not none %}
audita:
{% if narratio_audita_binary is not none %}
binary: {{ narratio_audita_binary | to_json }}
@@ -122,29 +122,20 @@ audita:
{% if narratio_audita_timeout is not none %}
timeout: {{ narratio_audita_timeout | to_json }}
{% endif %}
{% if narratio_audita_llm_api_key is not none %}
llm_api_key: {{ narratio_audita_llm_api_key | to_json }}
{% if narratio_audita_credentials is not none %}
credentials: {{ narratio_audita_credentials | to_json }}
{% endif %}
{% if narratio_audita_modules is not none %}
modules: {{ narratio_audita_modules | to_json }}
{% endif %}
{% if narratio_audita_base_url is not none %}
base_url: {{ narratio_audita_base_url | to_json }}
{% if narratio_audita_overlay is not none %}
overlay: {{ narratio_audita_overlay | to_json }}
{% endif %}
{% if narratio_audita_model is not none %}
model: {{ narratio_audita_model | to_json }}
{% if narratio_audita_proposal_profile is not none %}
proposal_profile: {{ narratio_audita_proposal_profile | to_json }}
{% endif %}
{% if narratio_audita_total_llm_concurrency is not none %}
total_llm_concurrency: {{ narratio_audita_total_llm_concurrency | to_json }}
{% endif %}
{% if narratio_audita_proposal_llm_concurrency is not none %}
proposal_llm_concurrency: {{ narratio_audita_proposal_llm_concurrency | to_json }}
{% endif %}
{% if narratio_audita_validation_model is not none %}
validation_model: {{ narratio_audita_validation_model | to_json }}
{% endif %}
{% if narratio_audita_validation_llm_concurrency is not none %}
validation_llm_concurrency: {{ narratio_audita_validation_llm_concurrency | to_json }}
{% if narratio_audita_validation_profile is not none %}
validation_profile: {{ narratio_audita_validation_profile | to_json }}
{% endif %}
{% if narratio_audita_transcript_description is not none %}
transcript_description: {{ narratio_audita_transcript_description | to_json }}
@@ -155,12 +146,6 @@ audita:
{% if narratio_audita_output_schema is not none %}
output_schema: {{ narratio_audita_output_schema | to_json }}
{% endif %}
{% if narratio_audita_work_dir_retention is not none %}
work_dir_retention: {{ narratio_audita_work_dir_retention | to_json }}
{% endif %}
{% if narratio_audita_report is not none %}
report: {{ narratio_audita_report | to_json }}
{% endif %}
{% endif %}
{% if narratio_normalize_output_path is not none or narratio_normalize_output_schema is not none or narratio_normalize_report is not none %}
normalize:
@@ -253,7 +238,7 @@ scriptorium:
artifact_families: {{ narratio_scriptorium_artifact_families | to_json }}
{% endif %}
{% endif %}
{% if narratio_notarius_enabled is not none or narratio_notarius_binary is not none or narratio_notarius_config_path is not none or narratio_notarius_config_profile is not none or narratio_notarius_pipeline_id is not none or narratio_notarius_timeout is not none or narratio_notarius_working_directory is not none or narratio_notarius_llm_api_key is not none or narratio_notarius_resume is not none or narratio_notarius_chunk_cache is not none or narratio_notarius_references is not none or narratio_notarius_outputs is not none %}
{% if narratio_notarius_enabled is not none or narratio_notarius_binary is not none or narratio_notarius_config_path is not none or narratio_notarius_config_overlay is not none or narratio_notarius_pipeline_id is not none or narratio_notarius_timeout is not none or narratio_notarius_working_directory is not none or narratio_notarius_credentials is not none or narratio_notarius_resume is not none or narratio_notarius_chunk_cache is not none or narratio_notarius_references is not none or narratio_notarius_outputs is not none %}
notarius:
{% if narratio_notarius_enabled is not none %}
enabled: {{ narratio_notarius_enabled | to_json }}
@@ -264,8 +249,8 @@ notarius:
{% if narratio_notarius_config_path is not none %}
config_path: {{ narratio_notarius_config_path | to_json }}
{% endif %}
{% if narratio_notarius_config_profile is not none %}
config_profile: {{ narratio_notarius_config_profile | to_json }}
{% if narratio_notarius_config_overlay is not none %}
config_overlay: {{ narratio_notarius_config_overlay | to_json }}
{% endif %}
{% if narratio_notarius_pipeline_id is not none %}
pipeline_id: {{ narratio_notarius_pipeline_id | to_json }}
@@ -276,8 +261,8 @@ notarius:
{% if narratio_notarius_working_directory is not none %}
working_directory: {{ narratio_notarius_working_directory | to_json }}
{% endif %}
{% if narratio_notarius_llm_api_key is not none %}
llm_api_key: {{ narratio_notarius_llm_api_key | to_json }}
{% if narratio_notarius_credentials is not none %}
credentials: {{ narratio_notarius_credentials | to_json }}
{% endif %}
{% if narratio_notarius_resume is not none %}
resume: {{ narratio_notarius_resume | to_json }}

View File

@@ -18,7 +18,10 @@ verify_baseline="$repository_root/tests/verify-baseline.yml"
representative="$repository_root/tests/representative.yml"
verify_config="$repository_root/tests/verify-config.yml"
ansible_playbook="$workspace_venv/bin/ansible-playbook"
baseline_args=(-e 'narratio_whisperx_transcribe_url=http://127.0.0.1:8000/transcribe')
baseline_args=(
-e 'narratio_whisperx_transcribe_url=http://127.0.0.1:8000/transcribe'
-e '{"narratio_audita_credentials":null,"narratio_audita_overlay":null,"narratio_audita_proposal_profile":null,"narratio_audita_validation_profile":null,"narratio_notarius_credentials":null,"narratio_notarius_config_overlay":null}'
)
cleanup() {
exit_status=$?
@@ -86,19 +89,27 @@ require_clean_recap "$recovery_repeat_log"
representative_log="$test_tmp/representative.log"
"$ansible_playbook" -i "$inventory" "$representative" | tee "$representative_log"
synthetic_notarius_api_key=notarius-representative-synthetic-value
synthetic_audita_api_key=audita-representative-synthetic-value
representative_validation_log="$test_tmp/representative-validation.log"
NARRATIO_TEST_NOTARIUS_API_KEY="$synthetic_notarius_api_key" \
NARRATIO_TEST_AUDITA_API_KEY="$synthetic_audita_api_key" \
"$ansible_playbook" -i "$inventory" "$verify_config" | tee "$representative_validation_log"
! grep -Fq "$synthetic_notarius_api_key" "$representative_log" "$representative_validation_log"
! grep -Fq "$synthetic_audita_api_key" "$representative_log" "$representative_validation_log"
representative_repeat_log="$test_tmp/representative-repeat.log"
"$ansible_playbook" -i "$inventory" "$representative" | tee "$representative_repeat_log"
require_clean_recap "$representative_repeat_log"
empty_profile_log="$test_tmp/empty-profile.log"
empty_overlay_log="$test_tmp/empty-overlay.log"
empty_inputs='{"narratio_audita_credentials":[],"narratio_notarius_credentials":[],"narratio_audita_overlay":"","narratio_audita_proposal_profile":"","narratio_audita_validation_profile":"","narratio_notarius_config_overlay":""}'
empty_expectations='{"narratio_test_audita_credentials":[],"narratio_test_notarius_credentials":[],"narratio_test_audita_overlay":"","narratio_test_audita_proposal_profile":"","narratio_test_audita_validation_profile":"","narratio_test_notarius_config_overlay":""}'
"$ansible_playbook" -i "$inventory" \
-e '{"narratio_notarius_config_profile": ""}' \
"$representative" | tee "$empty_profile_log"
NARRATIO_TEST_NOTARIUS_API_KEY="$synthetic_notarius_api_key" \
"$ansible_playbook" -i "$inventory" \
-e '{"narratio_test_notarius_config_profile": ""}' \
-e "$empty_inputs" \
"$representative" | tee "$empty_overlay_log"
"$ansible_playbook" -i "$inventory" \
-e "$empty_expectations" \
"$verify_config"
empty_repeat_log="$test_tmp/empty-repeat.log"
"$ansible_playbook" -i "$inventory" -e "$empty_inputs" \
"$representative" | tee "$empty_repeat_log"
require_clean_recap "$empty_repeat_log"

View File

@@ -24,9 +24,15 @@
narratio_seriatim_coalesce_gap: 0.0
narratio_seriatim_report: false
narratio_audita_modules: []
narratio_audita_report: false
narratio_notarius_llm_api_key: NARRATIO_TEST_NOTARIUS_API_KEY
narratio_notarius_config_profile: representative-profile
narratio_audita_credentials:
- NARRATIO_TEST_AUDITA_API_KEY
- NARRATIO_TEST_NOTARIUS_API_KEY
narratio_audita_overlay: representative-audita-overlay
narratio_audita_proposal_profile: representative-proposal
narratio_audita_validation_profile: representative-validation
narratio_notarius_credentials:
- NARRATIO_TEST_NOTARIUS_API_KEY
narratio_notarius_config_overlay: representative-overlay
narratio_notarius_resume: false
narratio_notarius_chunk_cache: auto
narratio_normalize_output_path: null

View File

@@ -9,9 +9,10 @@
src: /usr/local/etc/narratio/pipeline.yml
register: narratio_baseline_pipeline
- name: Verify null Notarius fields are omitted
- name: Verify null downstream fields are omitted
ansible.builtin.assert:
that:
- "'audita' not in narratio_pipeline"
- "'notarius' not in narratio_pipeline"
vars:
narratio_pipeline: "{{ narratio_baseline_pipeline.content | b64decode | from_yaml }}"

View File

@@ -24,9 +24,12 @@
- narratio_pipeline.seriatim.coalesce_gap == 0.0
- narratio_pipeline.seriatim.report == false
- narratio_pipeline.audita.modules == []
- narratio_pipeline.audita.report == false
- narratio_pipeline.notarius.llm_api_key == 'NARRATIO_TEST_NOTARIUS_API_KEY'
- narratio_pipeline.notarius.config_profile == narratio_expected_notarius_config_profile
- narratio_pipeline.audita.credentials == narratio_expected_audita_credentials
- narratio_pipeline.audita.overlay == narratio_expected_audita_overlay
- narratio_pipeline.audita.proposal_profile == narratio_expected_audita_proposal_profile
- narratio_pipeline.audita.validation_profile == narratio_expected_audita_validation_profile
- narratio_pipeline.notarius.credentials == narratio_expected_notarius_credentials
- narratio_pipeline.notarius.config_overlay == narratio_expected_notarius_config_overlay
- narratio_pipeline.notarius.resume == false
- narratio_pipeline.notarius.chunk_cache == 'auto'
- narratio_pipeline.trim.enabled == false
@@ -40,7 +43,14 @@
- "'normalize' not in narratio_pipeline"
vars:
narratio_pipeline: "{{ narratio_managed_pipeline.content | b64decode | from_yaml }}"
narratio_expected_notarius_config_profile: "{{ narratio_test_notarius_config_profile | default('representative-profile') }}"
narratio_expected_audita_credentials: >-
{{ narratio_test_audita_credentials |
default(['NARRATIO_TEST_AUDITA_API_KEY', 'NARRATIO_TEST_NOTARIUS_API_KEY']) }}
narratio_expected_audita_overlay: "{{ narratio_test_audita_overlay | default('representative-audita-overlay') }}"
narratio_expected_audita_proposal_profile: "{{ narratio_test_audita_proposal_profile | default('representative-proposal') }}"
narratio_expected_audita_validation_profile: "{{ narratio_test_audita_validation_profile | default('representative-validation') }}"
narratio_expected_notarius_credentials: "{{ narratio_test_notarius_credentials | default(['NARRATIO_TEST_NOTARIUS_API_KEY']) }}"
narratio_expected_notarius_config_overlay: "{{ narratio_test_notarius_config_overlay | default('representative-overlay') }}"
- name: Validate the representative pipeline as the Narratio account
become: true
@@ -54,4 +64,5 @@
- /usr/local/etc/narratio/pipeline.yml
environment:
NARRATIO_TEST_NOTARIUS_API_KEY: "{{ lookup('ansible.builtin.env', 'NARRATIO_TEST_NOTARIUS_API_KEY') }}"
NARRATIO_TEST_AUDITA_API_KEY: "{{ lookup('ansible.builtin.env', 'NARRATIO_TEST_AUDITA_API_KEY') }}"
changed_when: false