Add external facts of top scope vars

CLAUDE.md

@@ -0,0 +1,155 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## What This Repo Is
+
+PSICK (Puppet Systems Infrastructure Construction Kit) is a **Puppet control-repo** — the Git repository that a Puppet server deploys to manage infrastructure. It is not a standalone application; it is meant to be deployed to `/etc/puppetlabs/code/environments/<branch>/` on a Puppet server, where each git branch maps to a Puppet environment.
+
+## Setup
+
+Install prerequisite gems and populate `modules/` via r10k:
+
+```bash
+bin/puppet_setup.sh        # installs gems + runs r10k
+# or if gems already installed:
+r10k puppetfile install -v
+```
+
+For optional tools (Vagrant, Docker):
+
+```bash
+bin/setup.sh
+```
+
+## Testing Commands
+
+### Syntax Checks
+
+```bash
+bin/puppet_check_syntax_fast.sh all_but_chars   # validate manifests, templates, ruby files
+bin/puppet_check_syntax_fast.sh chars           # check for non-printable characters
+bin/puppet_lint.sh                              # run puppet-lint
+```
+
+### Unit Tests (Rake / onceover)
+
+```bash
+bundle exec rake spec                           # run rspec unit tests for the control-repo
+bin/puppet_check_rake.sh controlrepo           # same via helper script
+bin/puppet_check_rake.sh site                  # rspec tests for modules in site/
+bin/puppet_check_rake.sh site bundle           # with bundle install first
+```
+
+### onceover (catalog compilation tests)
+
+onceover compiles catalogs for classes against factsets, without a live Puppet server:
+
+```bash
+bundle exec onceover run spec                   # run onceover spec tests
+./run_onceover.sh run spec                      # same via podman container (puppet/puppet-dev-tools:4.x)
+```
+
+onceover config is in `spec/onceover.yaml`. Factsets (mock node facts) are in `spec/factsets/`.
+
+### Docker Integration Tests
+
+```bash
+bin/docker_run.sh --image centos-7             # test Puppet apply inside a Docker container
+```
+
+### Vagrant Integration Tests
+
+```bash
+cd vagrant/environments/lab   # (or ostest, bare, foss, pe)
+vagrant status
+vagrant up <vm>
+vagrant provision <vm>         # re-run Puppet apply on a running VM
+```
+
+## Architecture and Key Concepts
+
+### Nodeless, Hiera-Driven Classification
+
+`manifests/site.pp` includes only `include 'psick'`. The psick module drives all classification. Node roles are determined by **top-scope variables** set from trusted facts, external facts, or an ENC:
+
+- `$role` — maps to a Puppet role (e.g., `webserver`, `dbserver`)
+- `$env` — operational environment (e.g., `production`, `test`)
+- `$zone` / `$datacenter` / `$application` — optional further grouping
+
+These variables are used as keys in `hiera.yaml` to determine which Hiera data applies to a node.
+
+### Hiera Hierarchy (`hiera.yaml`)
+
+```
+nodes/%{trusted.certname}.yaml        # per-node override (highest priority)
+role/%{::role}-%{::env}.yaml          # role+environment combo
+role/%{::role}.yaml                    # role-level data
+zone/%{::zone}.yaml                    # zone/datacenter data
+common.yaml                            # global defaults (lowest priority)
+```
+
+Data lives in `modules/hieradata/data/` (the `example42/hieradata` module, sourced from the `psick-hieradata` git repo, branch matching the control branch). Hiera uses the eyaml backend for encrypted secrets; keys are at `/etc/puppetlabs/puppet/keys/`.
+
+`hiera_pabawi.yaml` is an alternate Hiera config that uses `facts.fact_role` instead of top-scope `$role` — for use with Pabawi/HDM (Hiera Data Manager) tools that resolve Hiera without catalog compilation.
+
+### Module Path (`environment.conf`)
+
+```
+modulepath = site:modules:$basemodulepath
+```
+
+- `site/` — local profiles and custom modules (in this repo)
+- `modules/` — external modules installed by r10k from `Puppetfile` (gitignored)
+- `$basemodulepath` — Puppet's built-in modules
+
+### Key External Modules (`Puppetfile`)
+
+- `example42/psick` — provides node classification and base profiles
+- `example42/psick_profile` — application-specific profiles
+- `example42/tp` + `example42/tinydata` — Tiny Puppet (generic application management)
+- `example42/hieradata` — Hiera data (sourced from psick-hieradata repo, branch-tracked)
+- `trlinkin/noop` — enables noop mode via Hiera key `noop_mode: true`
+
+### Noop Mode
+
+Set `noop_mode: true` in Hiera data to put nodes into noop (dry-run) mode without touching agent configuration.
+
+### Bolt (`bolt-project.yaml`)
+
+Bolt tasks and plans can be run against nodes. The project is named `psick` with `hiera.yaml` used for Hiera data. Available tasks include `tp::test` and `psick::puppet_*`.
+
+## CI/CD Pipelines
+
+### GitLab CI (`.gitlab-ci.yml`)
+
+Stages: `merge request → checks → unit → integration → canary → promote → production runs → documentation`
+
+Key scripts used by CI:
+- `bin/gitlab_before.sh <branch>` — setup (r10k deploy, etc.) before test stages
+- `bin/puppet_check_syntax_fast.sh` — syntax stage
+- `bin/puppet_check_rake.sh controlrepo` — unit stage
+- `bin/docker_run.sh` — Docker integration stage
+- `bin/puppet_ci.sh` — orchestrates PE/PuppetDB tasks (catalog preview, job runs)
+
+### Jenkins (`Jenkinsfile`)
+
+Stages: Setup → Syntax → Unit → Diff → Integration → Integration Rollout → Production Rollout
+
+## Directory Reference
+
+| Path | Purpose |
+|------|---------|
+| `manifests/site.pp` | Entry point; sets top-scope vars, includes psick |
+| `hiera.yaml` | Hiera 5 environment config with eyaml backend |
+| `hiera_pabawi.yaml` | Alternate Hiera config for HDM/Pabawi (uses facts instead of top-scope vars) |
+| `Puppetfile` | External module definitions for r10k |
+| `environment.conf` | Puppet environment settings (modulepath, manifest dir) |
+| `site/` | Local profiles and custom Puppet modules |
+| `modules/` | r10k-installed external modules (gitignored) |
+| `spec/onceover.yaml` | onceover test matrix (classes × nodes) |
+| `spec/factsets/` | Mock fact sets for onceover catalog tests |
+| `vagrant/environments/` | Vagrant test environments (lab, ostest, bare, foss, pe) |
+| `bin/` | Helper scripts for syntax checks, CI, Docker, Vagrant, r10k |
+| `docs/` | Extended documentation |
+| `bolt-project.yaml` | Bolt project config |

Puppetfile

@@ -9,21 +9,22 @@ mod 'example42/hieradata',
   :branch => :control_branch,
   :default_branch => 'production'
 
-# Puppet 6 Core Modues
-mod 'puppetlabs/mount_core', :latest
-mod 'puppetlabs/augeas_core', :latest
-mod 'puppetlabs/zfs_core', :latest
-mod 'puppetlabs/yumrepo_core', :latest
+# Puppet 6 Core Modules
+# They are included in official puppet/openvox agent packages
+# mod 'puppetlabs/mount_core', :latest
+# mod 'puppetlabs/augeas_core', :latest
+# mod 'puppetlabs/zfs_core', :latest
+# mod 'puppetlabs/yumrepo_core', :latest
 mod 'puppetlabs/host_core', :latest
-mod 'puppetlabs/selinux_core', :latest
-mod 'puppetlabs/zone_core', :latest
-mod 'puppetlabs/cron_core', :latest
-mod 'puppetlabs/sshkeys_core', :latest
-mod 'puppetlabs/nagios_core', :latest
-mod 'puppetlabs/mailalias_core', :latest
-mod 'puppetlabs/macdslocal_core', :latest
-mod 'puppetlabs/maillist_core', :latest
-mod 'puppetlabs/k5login_core', :latest
+# mod 'puppetlabs/selinux_core', :latest
+# mod 'puppetlabs/zone_core', :latest
+# mod 'puppetlabs/cron_core', :latest
+# mod 'puppetlabs/sshkeys_core', :latest
+# mod 'puppetlabs/nagios_core', :latest
+# mod 'puppetlabs/mailalias_core', :latest
+# mod 'puppetlabs/macdslocal_core', :latest
+# mod 'puppetlabs/maillist_core', :latest
+# mod 'puppetlabs/k5login_core', :latest
 
 # Example42 modules
 # From Forge
@@ -36,12 +37,12 @@ mod 'example42/psick_profile', :latest
 mod 'puppetlabs/concat', :latest
 mod 'puppetlabs/stdlib', :latest
 mod 'puppetlabs/vcsrepo', :latest
-mod 'puppetlabs/firewall', :latest
+# mod 'puppetlabs/firewall', :latest
 mod 'puppetlabs/inifile', :latest
-mod 'jdowning/rbenv', :latest
+# mod 'jdowning/rbenv', :latest
 mod 'trlinkin/noop', :latest
 mod 'puppet/archive', :latest
-mod 'puppetlabs-dropsonde', :latest
+# mod 'puppetlabs-dropsonde', :latest
 
 
 # Optionally used by psick_profile::openvpn
@@ -51,6 +52,7 @@ mod 'puppetlabs-dropsonde', :latest
 # mod 'puppetlabs/aws', :latest
 
 # Used by psick::puppet::foss_server
+
 # mod 'puppetlabs-bolt_shim', '0.3.0'
 # mod 'puppetlabs/postgresql', :latest
 # mod 'puppetlabs/puppetdb', :latest
@@ -71,23 +73,23 @@ mod 'puppetlabs-dropsonde', :latest
 #mod 'dwerder/graphite', :latest
 
 # Docker and Containers
-mod 'puppetlabs/dummy_service', :latest
+# mod 'puppetlabs/dummy_service', :latest
 #mod 'puppetlabs/image_build', :latest
 #mod 'puppetlabs/rkt', :latest
 
 # mod 'herculesteam-augeasproviders_sysctl', '2.2.0'
 # mod 'puppetlabs/firewall', :latest
 
 # Used by psick_profile::vagrant
-mod 'unibet/vagrant', :latest
+# mod 'unibet/vagrant', :latest
 
 # Used by psick::icinga
-mod 'icinga/icinga2', :latest
+# mod 'icinga/icinga2', :latest
 
 # Used by psick_profile::sensu
-mod 'sensu/sensu', :latest
-mod 'yelp/uchiwa', :latest
-mod 'puppet/rabbitmq', :latest
+# mod 'sensu/sensu', :latest
+# mod 'yelp/uchiwa', :latest
+# mod 'puppet/rabbitmq', :latest
 # deprecated: mod 'puppet/staging', :latest
 
 # Used by Windows profiles

hiera_pabawi.yaml

@@ -0,0 +1,15 @@
+version: 5
+defaults:
+  datadir: modules/hieradata/data
+hierarchy:
+  - name: Eyaml hierarchy
+    lookup_key: eyaml_lookup_key
+    paths:
+      - "nodes/%{trusted.certname}.yaml"
+      - "role/%{facts.fact_role}-%{facts.fact_env}.yaml"
+      - "role/%{facts.fact_role}.yaml"
+      - "zone/%{facts.fact_zone}.yaml"
+      - common.yaml
+    options:
+      pkcs7_private_key: /etc/puppetlabs/puppet/keys/private_key.pkcs7.pem
+      pkcs7_public_key: /etc/puppetlabs/puppet/keys/public_key.pkcs7.pem

manifests/site.pp

@@ -11,33 +11,52 @@
 
 ### SETTING TOP SCOPE VARIABLES USED IN HIERA.YAML
 # The following lines are used to assign to top-scope variables (used in
-# hiera.yaml) the values of eventual trusted facts.
-# More info: https://docs.puppet.com/puppet/latest/reference/ssl_attributes_extensions.html
+# hiera.yaml) the values of eventual trusted facts or relevant Hiera keys
+# if a fact for them is not already set.
 # You may need to change and adapt them according to your hiera.yaml
 # You can keep them also if you don't set extended trusted facts.
+
 if defined('$facts') and defined('$trusted') {
-  if $trusted['extensions']['pp_role'] and ! getvar('facts.role') {
-    $role = $trusted['extensions']['pp_role']
+  if ! getvar('facts.role') {
+    $role = pick_default(getvar('trusted.extensions.pp_role'), lookup('role', Optional[String], 'first', undef))
   }
-  if $trusted['extensions']['pp_environment'] and ! getvar('facts.env') {
-    $env = $trusted['extensions']['pp_environment']
+  if ! getvar('facts.env') {
+    $env = pick_default(getvar('trusted.extensions.pp_environment'), lookup('env', Optional[String], 'first', undef))
   }
-  if $trusted['extensions']['pp_datacenter'] and ! getvar('facts.datacenter') {
-    $datacenter = $trusted['extensions']['pp_datacenter']
+  if ! getvar('facts.datacenter') {
+    $datacenter =  pick_default(getvar('trusted.extensions.pp_datacenter'), lookup('datacenter', Optional[String], 'first', undef))
   }
-  if $trusted['extensions']['pp_zone'] and ! getvar('facts.zone') {
-    $zone = $trusted['extensions']['pp_zone']
+  if ! getvar('facts.zone') {
+    $zone = pick_default(getvar('trusted.extensions.pp_zone'), lookup('zone', Optional[String], 'first', undef))
   }
-  if $trusted['extensions']['pp_application'] and ! getvar('facts.application') {
-    $application = $trusted['extensions']['pp_application']
+  if ! getvar('facts.application') {
+    $application = pick_default(getvar('trusted.extensions.pp_application'), lookup('application', Optional[String], 'first', undef))
   }
   # Note: with the above settings we allow overriding of our trusted facts by normal facts.
   # This is done here to adapt to different approaches, if you use trusted facts
   # you will probably want to change the above into something like:
-  # if $trusted['extensions']['pp_role'] {
+  # if getvar('trusted.extensions.pp_role') {
   #   $role = $trusted['extensions']['pp_role']
   # }
 
+  # Creation of external facts with the values of the TopScope variables set here and used in hiera.yaml
+  # Useful for custom hiera_pabawi.yaml file to allow Hiera resolution without catalog compilation (in Pabawi or HDM)
+  psick::puppet::set_external_fact { 'fact_role':
+    value => $role,
+  }
+  psick::puppet::set_external_fact { 'fact_env':
+    value => $env,
+  }
+  psick::puppet::set_external_fact { 'fact_datacenter':
+    value => $datacenter,
+  }
+  psick::puppet::set_external_fact { 'fact_zone':
+    value => $zone,
+  }
+  psick::puppet::set_external_fact { 'fact_application':
+    value => $application,
+  }
+
 
   ### RESOURCE DEFAULTS
   # Some resource defaults for Files, Execs and Tiny Puppet
@@ -77,6 +96,7 @@
   ### ADDITIONS FOR RUNS INSIDE DOCKER IMAGES AND NOOP MODE
   # Building Docker container support
   # This has a fix for service provider on docker
+  # Uncomment mod 'puppetlabs/dummy_service', :latest in Puppetfile to make it working
   if $virtual == 'docker' {
     include ::dummy_service
   }
@@ -89,5 +109,5 @@
   }
 
   # We just do everything in psick module
-  include '::psick'
+  include 'psick'
 }

run_onceover.sh

@@ -0,0 +1,2 @@
+#!/bin/bash
+podman run --rm -v $(pwd):/repo --userns=keep-id puppet/puppet-dev-tools:4.x onceover "$@"