Skip to content

test(core,nativeshims): export sanity + interop KATs + Hkdf relocate#472

Merged
DennisDyallo merged 9 commits intodevelopfrom
feature/nativeshims-tests
Apr 29, 2026
Merged

test(core,nativeshims): export sanity + interop KATs + Hkdf relocate#472
DennisDyallo merged 9 commits intodevelopfrom
feature/nativeshims-tests

Conversation

@DennisDyallo
Copy link
Copy Markdown
Collaborator

@DennisDyallo DennisDyallo commented Apr 29, 2026

Summary

Yubico.NativeShims had no direct test coverage — CI verified the library built but never executed an exported function or validated per-platform export lists against the implementation. This PR adds that coverage and bundles related Core test-tree cleanups that surfaced while integrating with the parent webauthn previewSign branch (#468).

Stacking: merge before #468. The previewSign branch depends on this for the new export, the consumed-NuGet bump, and the relocated HkdfUtilities tests.

What's in the PR

  • Test infrastructureYubico.NativeShims/tests/check_exports.{sh,ps1} + expected_symbols.txt validate the export table on every per-platform build via build-nativeshims.yml. 30 xUnit P/Invoke KAT tests (BigNum / EcPoint / GcmEvp / Cmac) using NIST SP 800-38D, RFC 4493, SEC 1/2 vectors. build-macOS-local.sh for fast vcpkg-bypassing arm64 dev iteration.
  • New native exportNative_EC_POINT_is_on_curve (C function + per-platform export-list entries + managed wrapper + 2 on-curve tests). Consumed by ARKG-P256 on feat(fido2): WebAuthn previewSign extension (ARKG-P256) #468.
  • Consumed-NuGet bumpYubico.NativeShims 1.16.1-prerelease.20260428.1 so Core sees the new symbol at runtime.
  • Test-tree cleanup — moves HkdfUtilities from Yubico.YubiKey to Yubico.Core and relocates its tests to the Core test project to match the SUT's assembly. Adds spec-citation file headers (URLs verified resolvable + content-matching) on the new interop suites.

Verification

```
dotnet test Yubico.Core/tests/Yubico.Core.UnitTests.csproj \
--filter "FullyQualifiedNamePlatformInterop|FullyQualifiedNameHkdf"
```
58 / 58 PASS on macOS arm64. Export-table check verifies all 36 expected symbols against built binaries; removing any symbol from the canonical list correctly produces exit 1.

CI note

submit-nuget will fail until Yubico.NativeShims 1.16.1-prerelease.20260428.1 is reachable from the public NuGet feed (currently published only to Yubico's internal feed). Build + test CI checks across Windows/Linux/macOS all green.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 29, 2026

Test Results: Windows

    2 files      2 suites   41s ⏱️
4 117 tests 4 095 ✅ 22 💤 0 ❌
4 119 runs  4 097 ✅ 22 💤 0 ❌

Results for commit fc51473.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 29, 2026

Test Results: Ubuntu

    2 files      2 suites   1m 10s ⏱️
4 109 tests 4 087 ✅ 22 💤 0 ❌
4 111 runs  4 089 ✅ 22 💤 0 ❌

Results for commit fc51473.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 29, 2026

Test Results: MacOS

    4 files      4 suites   54s ⏱️
4 091 tests 4 088 ✅ 3 💤 0 ❌
4 093 runs  4 090 ✅ 3 💤 0 ❌

Results for commit fc51473.

♻️ This comment has been updated with latest results.

@DennisDyallo DennisDyallo force-pushed the feature/nativeshims-tests branch from 08d8f17 to 09091f8 Compare April 29, 2026 13:47
<PackageReference Include="System.Memory" Version="4.6.3" />
<PackageReference Include="System.Security.Principal.Windows" Version="5.0.0" />
<PackageReference Include="Yubico.NativeShims" Version="1.16.0" />
<PackageReference Include="Yubico.NativeShims" Version="1.16.1-prerelease.20260428.1" />
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm aware. Will be updated later

@DennisDyallo DennisDyallo changed the title test(nativeshims): add export-table sanity + P/Invoke KAT coverage test(core,nativeshims): export sanity + interop KATs + Hkdf relocate Apr 29, 2026
@DennisDyallo DennisDyallo requested a review from Copilot April 29, 2026 14:40
DennisDyallo and others added 7 commits April 29, 2026 16:49
Yubico.NativeShims previously had zero direct test coverage. The build CI
verified the library *built*, but never executed a single exported function
or validated that the per-platform export lists matched the implementation.
This change introduces two complementary layers and a local-dev iteration
script.

Layer A — export-table sanity (CI-blocking)
  * tests/expected_symbols.txt — single source of truth: the 35 Native_*
    symbols the library MUST export (BigNum, EC, GCM, CMAC, PCSC).
  * tests/check_exports.sh — POSIX validator using nm, runs on macOS and
    Linux; cross-arch safe (nm reads ELF metadata, works on arm64 binaries
    inspected from an x86_64 host).
  * tests/check_exports.ps1 — Windows validator using dumpbin; works on
    arm64 DLLs inspected from an x64 runner.
  * .github/workflows/build-nativeshims.yml — invokes the validator after
    every per-platform build (Windows x64/x86/arm64, Linux amd64/arm64,
    macOS x64/arm64). Build fails on missing or extra Native_* symbol —
    no more shipping a binary with a quietly-dropped export.

Layer B — managed P/Invoke functional tests (runs in existing PR workflows)
  * Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/{BigNum,
    EcPoint,GcmEvp,Cmac}InteropTests.cs — 30 xUnit tests using public
    Known-Answer Tests cited in comments:
      - NIST SP 800-38D (AES-GCM)
      - RFC 4493 (AES-128-CMAC)
      - SEC2 v2 §2.4.2 (P-256)
    Tests exercise the actual marshaling boundary consumers depend on:
    BigNum round-trips, EC point arithmetic / scalar mul, GCM tag tamper
    detection, CMAC multi-update equivalence. All deterministic, no
    hardware dependency. Runs in <100 ms.

Local dev
  * Yubico.NativeShims/build-macOS-local.sh — bypasses vcpkg using brew
    OpenSSL@3 for fast arm64 dev rebuilds; replaces the consumed NuGet
    cache dylib in-place. Now also runs check_exports.sh against the
    fresh build for immediate parity feedback.

Verification
  * dotnet test PlatformInterop filter: 30/30 passing locally
  * expected_symbols.txt aligned with origin/develop's exports.llvm
    (35/35 match)
  * Negative test: removing Native_BN_new from expected list produces
    exit 1 with diagnostic, as designed

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tion

Adds a single OpenSSL EC_POINT_is_on_curve wrapper to support ARKG-P256
input validation in Phase 3+. Untrusted public keys (pkBl, pkKem) received
from previewSign authenticator responses MUST be validated before use in
ARKG derivation to prevent invalid-curve attacks.

All 3 platform export files updated in alphabetical position. Native build
verification deferred to a follow-up step (VCPKG_INSTALLATION_ROOT not set
in this session); P/Invoke wrapper in Phase 3 will compile but will throw
EntryPointNotFoundException at runtime until the local NativeShims binary
is rebuilt and copied into ~/.nuget/packages/yubico.nativeshims/<ver>/.

Local build steps to run before Phase 3+ integration testing:
  cd Yubico.NativeShims && ./build-macOS.sh
  cp ./osx-x64/libYubico.NativeShims.dylib   \\
     ~/.nuget/packages/yubico.nativeshims/<ver>/runtimes/osx-x64/native/
  cp ./osx-arm64/libYubico.NativeShims.dylib \\
     ~/.nuget/packages/yubico.nativeshims/<ver>/runtimes/osx-arm64/native/

(VCPKG_INSTALLATION_ROOT must be set to a vcpkg source tree first.)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the managed P/Invoke wrapper (NativeMethods.EcPointIsOnCurve) and
two on-curve test cases (P-256 generator on curve; bit-flipped Y off curve)
that exercise the C function exposed by the prior commit. Updates
expected_symbols.txt to include Native_EC_POINT_is_on_curve so Layer A's
check_exports validator passes against the built library.

This keeps the PR self-contained: the new export, its managed wrapper, the
canonical-list entry, and the tests covering it all land together. Consumers
of the wrapper (ArkgPrimitivesOpenSsl, etc.) live on the webauthn previewSign
branch and depend on this groundwork.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Each of the four PlatformInterop crypto test files now opens with a Purpose,
What this validates, and References block stating the standards behind every
test (NIST SP 800-38D for GCM, RFC 4493 for CMAC, SEC 1/SEC 2/FIPS 186-5 for
P-256, OpenSSL man pages for the BIGNUM API). Reviewers and future
contributors can now see at a glance why the file exists, which entry points
it pins, and where to verify the test vectors come from.

No production code or test logic changes; comment-only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HKDF-SHA256 is a generic primitive that does not belong in the YubiKey
assembly. Moving it to Core enables Core-internal callers (notably the
ARKG implementation in ArkgPrimitivesOpenSsl) to share a single
canonical Span-based implementation instead of carrying a private copy.

Replaces the string-based CryptographyProviders.HmacCreator indirection
with direct BCL HMACSHA256 instantiation. The HmacCreator pattern was
not used anywhere outside HkdfUtilities for HMAC swap-ability (no test
overrides, no FIPS injection), so this removes a layer with no current
swap-need.

Bundles a security fix: the intermediate pseudo-random key from HKDF
extract is now zeroed via CryptographicOperations.ZeroMemory in a
try/finally before the function returns. Previously leaked.

Existing YubiKey callers updated to import the new namespace (AuthenticatorInfo
+ HkdfUtilitiesTests); behavior unchanged. KAT vectors verify byte-for-byte
equivalence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the test/SUT assembly mismatch left by the prior commit (75409d2)
which moved HkdfUtilities source from Yubico.YubiKey to Yubico.Core but kept
the test file in Yubico.YubiKey/tests with the namespace still pointing at
Yubico.YubiKey.Cryptography. Tests should mirror their SUT's assembly so
future refactors stay coherent.

Changes:
  * git mv Yubico.YubiKey/tests/.../HkdfUtilitiesTests.cs
        -> Yubico.Core/tests/unit/Yubico/Core/Cryptography/HkdfUtilitiesTests.cs
  * Namespace: Yubico.YubiKey.Cryptography -> Yubico.Core.Cryptography

No test logic changes. RFC 5869 KAT vectors unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@DennisDyallo DennisDyallo force-pushed the feature/nativeshims-tests branch from 059659c to 8f9e56c Compare April 29, 2026 14:50
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR strengthens the SDK’s native/managed cryptography integration by adding export-table verification for Yubico.NativeShims, introducing P/Invoke known-answer tests (KATs) for key OpenSSL wrappers, adding a new EC-point on-curve native export + managed wrapper, and relocating HKDF utilities/tests into Yubico.Core to match usage.

Changes:

  • Add canonical export-symbol validation scripts and wire them into the NativeShims build workflow across Windows/Linux/macOS.
  • Add new Native_EC_POINT_is_on_curve export plus managed P/Invoke wrapper and interop tests.
  • Move HkdfUtilities into Yubico.Core (and adjust tests/usages), and bump the consumed Yubico.NativeShims NuGet version.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Yubico.YubiKey/src/Yubico/YubiKey/Fido2/AuthenticatorInfo.cs Switch HKDF usage to Core cryptography namespace after HKDF relocation.
Yubico.NativeShims/tests/expected_symbols.txt Define canonical exported-symbol list for NativeShims.
Yubico.NativeShims/tests/check_exports.sh POSIX export-table verifier using nm + canonical list.
Yubico.NativeShims/tests/check_exports.ps1 Windows export-table verifier using dumpbin + canonical list.
Yubico.NativeShims/ssl.ecpoint.c Add Native_EC_POINT_is_on_curve export implementation.
Yubico.NativeShims/exports.msvc Add new export to Windows .def list.
Yubico.NativeShims/exports.llvm Add new export to macOS export list.
Yubico.NativeShims/exports.gnu Add new export to GNU/Linux version script.
Yubico.NativeShims/build-macOS-local.sh Add local macOS arm64 build helper that also runs export checks.
Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/GcmEvpInteropTests.cs Add AES-256-GCM EVP wrapper interop KATs + tamper checks.
Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/EcPointInteropTests.cs Add EC group/point lifecycle + mul + on-curve interop coverage.
Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/CmacInteropTests.cs Add RFC 4493 CMAC interop KATs + chunking equivalence tests.
Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/BigNumInteropTests.cs Add basic BN marshaling interop round-trip tests.
Yubico.Core/tests/unit/Yubico/Core/Cryptography/HkdfUtilitiesTests.cs Relocate HKDF tests to Core test project/namespace.
Yubico.Core/src/Yubico/PlatformInterop/Desktop/Cryptography/EcPoint.Interop.cs Add managed P/Invoke wrapper for Native_EC_POINT_is_on_curve.
Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs Relocate HKDF utility to Core and adjust implementation to use BCL HMACSHA256.
Yubico.Core/src/Yubico.Core.csproj Bump consumed Yubico.NativeShims package version.
.github/workflows/build-nativeshims.yml Run export-table checks as part of NativeShims CI builds.
Comments suppressed due to low confidence (6)

Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:87

  • In HkdfExpand, pseudoRandomKey.ToArray() creates an additional copy of the PRK that is not cleared, even though pseudoRandomKey itself is zeroed by the caller. Consider reusing/owning a single PRK byte[] for the HMAC key and zeroing it (and any intermediate blocks) when done.
    Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:20
  • The repository’s C# code in this folder consistently uses block-scoped namespaces (e.g., CmacPrimitives.cs). This file uses a file-scoped namespace, which makes the style inconsistent within Yubico.Core.Cryptography. Consider switching to the block-scoped namespace style for consistency.
    Yubico.Core/tests/unit/Yubico/Core/Cryptography/HkdfUtilitiesTests.cs:22
  • This test file is the only one in Yubico.Core tests using a file-scoped namespace (and it also has a redundant using for the same namespace). For consistency with the rest of the test suite (block-scoped namespaces), consider converting it to the same namespace style used elsewhere.
    Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:89
  • In HkdfExpand, the loop counter is a byte. When length == 8160, numberOfBlocks becomes 255 and the for-loop overflows from 255 back to 0, causing an infinite loop. Use an int loop variable and only cast to byte when appending the counter to the HMAC input.
    Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:22
  • HkdfUtilities is now public, but the type itself has no XML doc comment. With GenerateDocumentationFile enabled and warnings treated as errors, this will typically raise CS1591 for a public type. Add a for the class (and keep it consistent with other public crypto primitives).
    Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:73
  • HkdfExtract allocates copies of potentially sensitive material (salt.ToArray() and inputKeyMaterial.ToArray()) but never clears them. If this utility is intended for key-derivation with secret inputs, consider zeroing the temporary byte[] buffers after ComputeHash (similar to how pseudoRandomKey is cleared).

Comment thread Yubico.NativeShims/build-macOS-local.sh Outdated
Comment thread Yubico.Core/src/Yubico.Core.csproj
DennisDyallo and others added 2 commits April 29, 2026 16:53
@DennisDyallo DennisDyallo merged commit 3ed5998 into develop Apr 29, 2026
5 of 6 checks passed
@DennisDyallo DennisDyallo deleted the feature/nativeshims-tests branch April 29, 2026 15:07
@github-actions
Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Branch Rate Complexity Health
Yubico.Core 53% 48% 1584
Yubico.YubiKey 50% 45% 7180
Summary 51% (12922 / 25583) 46% (3162 / 6912) 8764

Minimum allowed line rate is 40%

@DennisDyallo DennisDyallo mentioned this pull request Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants