test(core,nativeshims): export sanity + interop KATs + Hkdf relocate#472
test(core,nativeshims): export sanity + interop KATs + Hkdf relocate#472DennisDyallo merged 9 commits intodevelopfrom
Conversation
Test Results: Windows 2 files 2 suites 41s ⏱️ Results for commit fc51473. ♻️ This comment has been updated with latest results. |
Test Results: Ubuntu 2 files 2 suites 1m 10s ⏱️ Results for commit fc51473. ♻️ This comment has been updated with latest results. |
Test Results: MacOS 4 files 4 suites 54s ⏱️ Results for commit fc51473. ♻️ This comment has been updated with latest results. |
08d8f17 to
09091f8
Compare
| <PackageReference Include="System.Memory" Version="4.6.3" /> | ||
| <PackageReference Include="System.Security.Principal.Windows" Version="5.0.0" /> | ||
| <PackageReference Include="Yubico.NativeShims" Version="1.16.0" /> | ||
| <PackageReference Include="Yubico.NativeShims" Version="1.16.1-prerelease.20260428.1" /> |
There was a problem hiding this comment.
I'm aware. Will be updated later
Yubico.NativeShims previously had zero direct test coverage. The build CI
verified the library *built*, but never executed a single exported function
or validated that the per-platform export lists matched the implementation.
This change introduces two complementary layers and a local-dev iteration
script.
Layer A — export-table sanity (CI-blocking)
* tests/expected_symbols.txt — single source of truth: the 35 Native_*
symbols the library MUST export (BigNum, EC, GCM, CMAC, PCSC).
* tests/check_exports.sh — POSIX validator using nm, runs on macOS and
Linux; cross-arch safe (nm reads ELF metadata, works on arm64 binaries
inspected from an x86_64 host).
* tests/check_exports.ps1 — Windows validator using dumpbin; works on
arm64 DLLs inspected from an x64 runner.
* .github/workflows/build-nativeshims.yml — invokes the validator after
every per-platform build (Windows x64/x86/arm64, Linux amd64/arm64,
macOS x64/arm64). Build fails on missing or extra Native_* symbol —
no more shipping a binary with a quietly-dropped export.
Layer B — managed P/Invoke functional tests (runs in existing PR workflows)
* Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/{BigNum,
EcPoint,GcmEvp,Cmac}InteropTests.cs — 30 xUnit tests using public
Known-Answer Tests cited in comments:
- NIST SP 800-38D (AES-GCM)
- RFC 4493 (AES-128-CMAC)
- SEC2 v2 §2.4.2 (P-256)
Tests exercise the actual marshaling boundary consumers depend on:
BigNum round-trips, EC point arithmetic / scalar mul, GCM tag tamper
detection, CMAC multi-update equivalence. All deterministic, no
hardware dependency. Runs in <100 ms.
Local dev
* Yubico.NativeShims/build-macOS-local.sh — bypasses vcpkg using brew
OpenSSL@3 for fast arm64 dev rebuilds; replaces the consumed NuGet
cache dylib in-place. Now also runs check_exports.sh against the
fresh build for immediate parity feedback.
Verification
* dotnet test PlatformInterop filter: 30/30 passing locally
* expected_symbols.txt aligned with origin/develop's exports.llvm
(35/35 match)
* Negative test: removing Native_BN_new from expected list produces
exit 1 with diagnostic, as designed
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tion
Adds a single OpenSSL EC_POINT_is_on_curve wrapper to support ARKG-P256
input validation in Phase 3+. Untrusted public keys (pkBl, pkKem) received
from previewSign authenticator responses MUST be validated before use in
ARKG derivation to prevent invalid-curve attacks.
All 3 platform export files updated in alphabetical position. Native build
verification deferred to a follow-up step (VCPKG_INSTALLATION_ROOT not set
in this session); P/Invoke wrapper in Phase 3 will compile but will throw
EntryPointNotFoundException at runtime until the local NativeShims binary
is rebuilt and copied into ~/.nuget/packages/yubico.nativeshims/<ver>/.
Local build steps to run before Phase 3+ integration testing:
cd Yubico.NativeShims && ./build-macOS.sh
cp ./osx-x64/libYubico.NativeShims.dylib \\
~/.nuget/packages/yubico.nativeshims/<ver>/runtimes/osx-x64/native/
cp ./osx-arm64/libYubico.NativeShims.dylib \\
~/.nuget/packages/yubico.nativeshims/<ver>/runtimes/osx-arm64/native/
(VCPKG_INSTALLATION_ROOT must be set to a vcpkg source tree first.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the managed P/Invoke wrapper (NativeMethods.EcPointIsOnCurve) and two on-curve test cases (P-256 generator on curve; bit-flipped Y off curve) that exercise the C function exposed by the prior commit. Updates expected_symbols.txt to include Native_EC_POINT_is_on_curve so Layer A's check_exports validator passes against the built library. This keeps the PR self-contained: the new export, its managed wrapper, the canonical-list entry, and the tests covering it all land together. Consumers of the wrapper (ArkgPrimitivesOpenSsl, etc.) live on the webauthn previewSign branch and depend on this groundwork. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Each of the four PlatformInterop crypto test files now opens with a Purpose, What this validates, and References block stating the standards behind every test (NIST SP 800-38D for GCM, RFC 4493 for CMAC, SEC 1/SEC 2/FIPS 186-5 for P-256, OpenSSL man pages for the BIGNUM API). Reviewers and future contributors can now see at a glance why the file exists, which entry points it pins, and where to verify the test vectors come from. No production code or test logic changes; comment-only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HKDF-SHA256 is a generic primitive that does not belong in the YubiKey assembly. Moving it to Core enables Core-internal callers (notably the ARKG implementation in ArkgPrimitivesOpenSsl) to share a single canonical Span-based implementation instead of carrying a private copy. Replaces the string-based CryptographyProviders.HmacCreator indirection with direct BCL HMACSHA256 instantiation. The HmacCreator pattern was not used anywhere outside HkdfUtilities for HMAC swap-ability (no test overrides, no FIPS injection), so this removes a layer with no current swap-need. Bundles a security fix: the intermediate pseudo-random key from HKDF extract is now zeroed via CryptographicOperations.ZeroMemory in a try/finally before the function returns. Previously leaked. Existing YubiKey callers updated to import the new namespace (AuthenticatorInfo + HkdfUtilitiesTests); behavior unchanged. KAT vectors verify byte-for-byte equivalence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the test/SUT assembly mismatch left by the prior commit (75409d2) which moved HkdfUtilities source from Yubico.YubiKey to Yubico.Core but kept the test file in Yubico.YubiKey/tests with the namespace still pointing at Yubico.YubiKey.Cryptography. Tests should mirror their SUT's assembly so future refactors stay coherent. Changes: * git mv Yubico.YubiKey/tests/.../HkdfUtilitiesTests.cs -> Yubico.Core/tests/unit/Yubico/Core/Cryptography/HkdfUtilitiesTests.cs * Namespace: Yubico.YubiKey.Cryptography -> Yubico.Core.Cryptography No test logic changes. RFC 5869 KAT vectors unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
059659c to
8f9e56c
Compare
There was a problem hiding this comment.
Pull request overview
This PR strengthens the SDK’s native/managed cryptography integration by adding export-table verification for Yubico.NativeShims, introducing P/Invoke known-answer tests (KATs) for key OpenSSL wrappers, adding a new EC-point on-curve native export + managed wrapper, and relocating HKDF utilities/tests into Yubico.Core to match usage.
Changes:
- Add canonical export-symbol validation scripts and wire them into the NativeShims build workflow across Windows/Linux/macOS.
- Add new
Native_EC_POINT_is_on_curveexport plus managed P/Invoke wrapper and interop tests. - Move
HkdfUtilitiesintoYubico.Core(and adjust tests/usages), and bump the consumedYubico.NativeShimsNuGet version.
Reviewed changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| Yubico.YubiKey/src/Yubico/YubiKey/Fido2/AuthenticatorInfo.cs | Switch HKDF usage to Core cryptography namespace after HKDF relocation. |
| Yubico.NativeShims/tests/expected_symbols.txt | Define canonical exported-symbol list for NativeShims. |
| Yubico.NativeShims/tests/check_exports.sh | POSIX export-table verifier using nm + canonical list. |
| Yubico.NativeShims/tests/check_exports.ps1 | Windows export-table verifier using dumpbin + canonical list. |
| Yubico.NativeShims/ssl.ecpoint.c | Add Native_EC_POINT_is_on_curve export implementation. |
| Yubico.NativeShims/exports.msvc | Add new export to Windows .def list. |
| Yubico.NativeShims/exports.llvm | Add new export to macOS export list. |
| Yubico.NativeShims/exports.gnu | Add new export to GNU/Linux version script. |
| Yubico.NativeShims/build-macOS-local.sh | Add local macOS arm64 build helper that also runs export checks. |
| Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/GcmEvpInteropTests.cs | Add AES-256-GCM EVP wrapper interop KATs + tamper checks. |
| Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/EcPointInteropTests.cs | Add EC group/point lifecycle + mul + on-curve interop coverage. |
| Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/CmacInteropTests.cs | Add RFC 4493 CMAC interop KATs + chunking equivalence tests. |
| Yubico.Core/tests/unit/Yubico/PlatformInterop/Cryptography/BigNumInteropTests.cs | Add basic BN marshaling interop round-trip tests. |
| Yubico.Core/tests/unit/Yubico/Core/Cryptography/HkdfUtilitiesTests.cs | Relocate HKDF tests to Core test project/namespace. |
| Yubico.Core/src/Yubico/PlatformInterop/Desktop/Cryptography/EcPoint.Interop.cs | Add managed P/Invoke wrapper for Native_EC_POINT_is_on_curve. |
| Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs | Relocate HKDF utility to Core and adjust implementation to use BCL HMACSHA256. |
| Yubico.Core/src/Yubico.Core.csproj | Bump consumed Yubico.NativeShims package version. |
| .github/workflows/build-nativeshims.yml | Run export-table checks as part of NativeShims CI builds. |
Comments suppressed due to low confidence (6)
Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:87
- In HkdfExpand, pseudoRandomKey.ToArray() creates an additional copy of the PRK that is not cleared, even though pseudoRandomKey itself is zeroed by the caller. Consider reusing/owning a single PRK byte[] for the HMAC key and zeroing it (and any intermediate blocks) when done.
Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:20 - The repository’s C# code in this folder consistently uses block-scoped namespaces (e.g., CmacPrimitives.cs). This file uses a file-scoped namespace, which makes the style inconsistent within Yubico.Core.Cryptography. Consider switching to the block-scoped namespace style for consistency.
Yubico.Core/tests/unit/Yubico/Core/Cryptography/HkdfUtilitiesTests.cs:22 - This test file is the only one in Yubico.Core tests using a file-scoped namespace (and it also has a redundant using for the same namespace). For consistency with the rest of the test suite (block-scoped namespaces), consider converting it to the same namespace style used elsewhere.
Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:89 - In HkdfExpand, the loop counter is a byte. When length == 8160, numberOfBlocks becomes 255 and the for-loop overflows from 255 back to 0, causing an infinite loop. Use an int loop variable and only cast to byte when appending the counter to the HMAC input.
Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:22 - HkdfUtilities is now public, but the type itself has no XML doc comment. With GenerateDocumentationFile enabled and warnings treated as errors, this will typically raise CS1591 for a public type. Add a
for the class (and keep it consistent with other public crypto primitives).
Yubico.Core/src/Yubico/Core/Cryptography/HkdfUtilities.cs:73 - HkdfExtract allocates copies of potentially sensitive material (salt.ToArray() and inputKeyMaterial.ToArray()) but never clears them. If this utility is intended for key-derivation with secret inputs, consider zeroing the temporary byte[] buffers after ComputeHash (similar to how pseudoRandomKey is cleared).
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Summary
Yubico.NativeShimshad no direct test coverage — CI verified the library built but never executed an exported function or validated per-platform export lists against the implementation. This PR adds that coverage and bundles related Core test-tree cleanups that surfaced while integrating with the parent webauthn previewSign branch (#468).Stacking: merge before #468. The previewSign branch depends on this for the new export, the consumed-NuGet bump, and the relocated
HkdfUtilitiestests.What's in the PR
Yubico.NativeShims/tests/check_exports.{sh,ps1}+expected_symbols.txtvalidate the export table on every per-platform build viabuild-nativeshims.yml. 30 xUnit P/Invoke KAT tests (BigNum / EcPoint / GcmEvp / Cmac) using NIST SP 800-38D, RFC 4493, SEC 1/2 vectors.build-macOS-local.shfor fast vcpkg-bypassing arm64 dev iteration.Native_EC_POINT_is_on_curve(C function + per-platform export-list entries + managed wrapper + 2 on-curve tests). Consumed by ARKG-P256 on feat(fido2): WebAuthn previewSign extension (ARKG-P256) #468.Yubico.NativeShims 1.16.1-prerelease.20260428.1so Core sees the new symbol at runtime.HkdfUtilitiesfromYubico.YubiKeytoYubico.Coreand relocates its tests to the Core test project to match the SUT's assembly. Adds spec-citation file headers (URLs verified resolvable + content-matching) on the new interop suites.Verification
```
dotnet test Yubico.Core/tests/Yubico.Core.UnitTests.csproj \
--filter "FullyQualifiedName
PlatformInterop|FullyQualifiedNameHkdf"```
→ 58 / 58 PASS on macOS arm64. Export-table check verifies all 36 expected symbols against built binaries; removing any symbol from the canonical list correctly produces exit 1.
CI note
submit-nugetwill fail untilYubico.NativeShims 1.16.1-prerelease.20260428.1is reachable from the public NuGet feed (currently published only to Yubico's internal feed). Build + test CI checks across Windows/Linux/macOS all green.