[WIP] Add MAUI Android inner loop deploy measurement scenario

.github/skills/maui-android-innerloop/SKILL.md

@@ -0,0 +1,123 @@
+---
+name: maui-android-innerloop
+description: Guide for running MAUI Android Inner Loop deploy measurements in the dotnet/performance repo. Use this when asked to measure, benchmark, or compare MAUI Android first deploy and incremental deploy times across runtime configurations (Mono+Interpreter, CoreCLR+JIT, etc.).
+---
+
+# MAUI Android Inner Loop Measurement
+
+Measures first deploy and incremental deploy times for a .NET MAUI Android app using MSBuild binary logs. Located in `src/scenarios/mauiandroidinnerloop/` within the dotnet/performance repo.
+
+## Prerequisites
+
+Before running measurements, verify:
+
+1. **Android device** connected via USB: `adb devices` should list a device.
+2. **.NET SDK** installed at `tools/dotnet/arm64/`. Bootstrap with:
+   ```bash
+   cd src/scenarios && . ./init.sh -channel main
+   ```
+3. **maui-android workload** installed (NOT full `maui` — iOS packages fail without the iOS workload):
+   ```bash
+   export DOTNET_ROOT="$(pwd)/tools/dotnet/arm64"
+   export PATH="$DOTNET_ROOT:$PATH"
+   dotnet workload install maui-android \
+     --from-rollback-file src/scenarios/mauiandroidinnerloop/rollback_maui.json \
+     --skip-sign-check
+   ```
+4. **Startup tool** (binlog parser) built:
+   ```bash
+   PERFLAB_TARGET_FRAMEWORKS=net11.0 dotnet publish \
+     src/tools/ScenarioMeasurement/Startup/Startup.csproj \
+     -c Release -o artifacts/startup --ignore-failed-sources /p:NuGetAudit=false
+   ```
+
+## Environment Setup (every new shell)
+
+```bash
+cd src/scenarios && . ./init.sh -dotnetdir <REPO_ROOT>/tools/dotnet/arm64
+cd mauiandroidinnerloop
+```
+
+## Create the App Template
+
+```bash
+python3 pre.py publish -f net11.0-android --has-workload
+```
+
+### CRITICAL: Fix csproj after pre.py
+
+`dotnet new maui` targets all platforms. Since only maui-android workload is installed, you MUST edit `app/MauiAndroidInnerLoop.csproj` and remove the iOS/MacCatalyst/Windows TargetFrameworks conditions, leaving only:
+
+```xml
+<TargetFrameworks>net11.0-android</TargetFrameworks>
+```
+
+Remove these two lines that follow the android TargetFrameworks line:
+```xml
+<TargetFrameworks Condition="!$([MSBuild]::IsOSPlatform('linux'))">$(TargetFrameworks);net11.0-ios;net11.0-maccatalyst</TargetFrameworks>
+<TargetFrameworks Condition="$([MSBuild]::IsOSPlatform('windows'))">$(TargetFrameworks);net11.0-windows10.0.19041.0</TargetFrameworks>
+```
+
+## MSBuild Args per Configuration
+
+| Configuration     | MSBuild Properties                                                                    |
+|-------------------|---------------------------------------------------------------------------------------|
+| Mono+Interpreter  | `/p:UseMonoRuntime=true`                                                              |
+| CoreCLR+JIT       | `/p:UseMonoRuntime=false /p:PublishReadyToRun=false /p:PublishReadyToRunComposite=false` |
+
+## Running a Measurement
+
+```bash
+# Clean from any prior run
+rm -rf app/bin app/obj traces
+
+# Mono+Interpreter
+python3 test.py androidinnerloop \
+  --csproj-path app/MauiAndroidInnerLoop.csproj \
+  --edit-src src/MainPage.xaml.cs \
+  --edit-dest app/MainPage.xaml.cs \
+  -f net11.0-android -c Debug \
+  --msbuild-args "/p:UseMonoRuntime=true"
+
+# CoreCLR+JIT
+python3 test.py androidinnerloop \
+  --csproj-path app/MauiAndroidInnerLoop.csproj \
+  --edit-src src/MainPage.xaml.cs \
+  --edit-dest app/MainPage.xaml.cs \
+  -f net11.0-android -c Debug \
+  --msbuild-args "/p:UseMonoRuntime=false;/p:PublishReadyToRun=false;/p:PublishReadyToRunComposite=false"
+```
+
+## What test.py androidinnerloop Does
+
+1. **First deploy:** `dotnet build <csproj> -t:Install -c Debug -f net11.0-android <msbuild-args> /p:UseSharedCompilation=true -bl:traces/first-deploy.binlog`
+2. **File edit:** Copies modified `MainPage.xaml.cs` to simulate an incremental code edit
+3. **Incremental deploy:** `dotnet build <csproj> -t:Install ... -bl:traces/incremental-deploy.binlog`
+4. **Parse binlogs:** Extracts per-task timings using the Startup tool
+
+## Clean Between Configurations
+
+```bash
+python3 post.py   # Uninstalls APK, shuts down build servers, removes app/traces/etc.
+python3 pre.py publish -f net11.0-android --has-workload
+# Fix csproj again! (remove iOS/MacCatalyst/Windows targets)
+```
+
+## Persisting Binlogs
+
+Binlogs are in `traces/` and get cleaned by `post.py`. To keep them:
+
+```bash
+mkdir -p binlogs
+cp traces/first-deploy.binlog binlogs/<config>-first-deploy.binlog
+cp traces/incremental-deploy.binlog binlogs/<config>-incremental-deploy.binlog
+```
+
+## Key Facts & Gotchas
+
+- **UseSharedCompilation=true** is set by `runner.py` for this scenario, overriding the repo default of `false`. This mirrors a real dev workflow where the Roslyn compiler server stays warm. `post.py` runs `dotnet build-server shutdown` to clean up between runs.
+- **FastDev** (Fast Deployment) is ON by default in Debug (`EmbedAssembliesIntoApk=false`). Do NOT set `EmbedAssembliesIntoApk=true` — it disables FastDev and makes deploys 10x slower.
+- **Startup tool** targets `net8.0` by default and needs .NET 8 runtime. Build with `PERFLAB_TARGET_FRAMEWORKS=net11.0` to retarget if only .NET 11 is available.
+- **Dead NuGet feeds** (`darc-pub-dotnet-android-*`) break Startup tool builds. Use `--ignore-failed-sources /p:NuGetAudit=false`.
+- **macOS Spotlight** can race with builds causing random errors (XARDF7024, MAUIR0001, CS2012). Fix: `sudo mdutil -i off <worktree_path>`.
+- **The repo sets `UseSharedCompilation=false`** in `src/Directory.Build.props` and `src/scenarios/init.sh`. The runner.py override via `/p:UseSharedCompilation=true` on the command line takes precedence.

eng/performance/maui_scenarios_android_innerloop.proj

@@ -0,0 +1,72 @@
+<Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">
+
+  <Import Project="Scenarios.Common.props" />
+
+  <PropertyGroup>
+    <IncludeXHarnessCli>true</IncludeXHarnessCli>
+  </PropertyGroup>
+
+  <PropertyGroup>
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono'">/p:UseMonoRuntime=true</_MSBuildArgs>
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false</_MSBuildArgs>
+
+    <!-- Mono AOT -->
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono' and '$(CodegenType)' == 'AOT'">$(_MSBuildArgs);/p:RunAOTCompilation=true;/p:AndroidEnableProfiledAot=false</_MSBuildArgs>
+    <!-- CoreCLR JIT -->
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr' and '$(CodegenType)' == 'JIT'">$(_MSBuildArgs);/p:PublishReadyToRun=false;/p:PublishReadyToRunComposite=false</_MSBuildArgs>
+    <!-- CoreCLR R2R -->
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr' and '$(CodegenType)' == 'R2R'">$(_MSBuildArgs);/p:PublishReadyToRun=true;/p:PublishReadyToRunComposite=false</_MSBuildArgs>
+    <!-- CoreCLR R2R composite -->
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr' and '$(CodegenType)' == 'R2RComposite'">$(_MSBuildArgs);/p:PublishReadyToRun=true;/p:PublishReadyToRunComposite=true</_MSBuildArgs>
+    <!-- CoreCLR NativeAOT -->
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr' and '$(CodegenType)' == 'NativeAOT'">$(_MSBuildArgs);/p:PublishAot=true</_MSBuildArgs>
+
+    <!-- Debug builds use Fast Deployment by default, which excludes assemblies from the APK. -->
+    <_MSBuildArgs Condition="'$(BuildConfig)' == 'Debug'">$(_MSBuildArgs);/p:EmbedAssembliesIntoApk=true</_MSBuildArgs>
+
+    <RunConfigsString>$(RuntimeFlavor)_$(CodegenType)</RunConfigsString>
+  </PropertyGroup>
+
+  <!--
+    NOTE: Unlike maui_scenarios_android.proj, we intentionally do NOT remove dotnet\packs
+    from the correlation staging. The inner loop scenario needs the full .NET SDK with
+    MAUI workload packs to build on the Helix machine.
+  -->
+
+  <ItemDefinitionGroup>
+    <HelixWorkItem>
+      <Timeout>01:00</Timeout>
+    </HelixWorkItem>
+  </ItemDefinitionGroup>
+
+  <ItemGroup>
+    <MAUIAndroidInnerLoopScenario Include="MAUI Android Inner Loop">
+      <ScenarioDirectoryName>mauiandroidinnerloop</ScenarioDirectoryName>
+      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
+    </MAUIAndroidInnerLoopScenario>
+  </ItemGroup>
+
+  <!-- PreparePayloadWorkItem: create template and modified source on the build machine.
+       pre.py creates the MAUI template in app/, fixes .csproj TFMs, and prepares the
+       modified MainPage.xaml.cs in src/ for the incremental deploy simulation. -->
+  <ItemGroup>
+    <PreparePayloadWorkItem Include="@(MAUIAndroidInnerLoopScenario)">
+      <Command>$(Python) pre.py</Command>
+      <WorkingDirectory>%(PreparePayloadWorkItem.PayloadDirectory)</WorkingDirectory>
+    </PreparePayloadWorkItem>
+  </ItemGroup>
+
+  <!-- HelixWorkItem: build and deploy on the Helix machine with an attached Android device.
+       test.py runs two dotnet build -t:Install commands (first deploy + incremental deploy
+       after a file edit) and parses the resulting binlogs for per-task timing data. -->
+  <ItemGroup>
+    <HelixWorkItem Include="@(MAUIAndroidInnerLoopScenario -> 'Inner Loop Deploy - %(Identity)')">
+      <PreCommands>set DOTNET_ROOT=%HELIX_CORRELATION_PAYLOAD%\dotnet&amp;&amp; set PATH=%HELIX_CORRELATION_PAYLOAD%\dotnet;%PATH%&amp;&amp; set NUGET_PACKAGES=%HELIX_WORKITEM_ROOT%\.packages</PreCommands>
+      <Command>$(Python) test.py androidinnerloop --csproj-path app\MauiAndroidInnerLoop.csproj --edit-src src\MainPage.xaml.cs --edit-dest app\MainPage.xaml.cs -f $(PERFLAB_Framework)-android -c $(BuildConfig) --msbuild-args &quot;$(_MSBuildArgs)&quot; --scenario-name &quot;%(Identity)&quot; $(ScenarioArgs)</Command>
+      <PostCommands>$(Python) post.py</PostCommands>
+    </HelixWorkItem>
+  </ItemGroup>
+
+  <Import Project="PreparePayloadWorkItems.targets" />
+
+</Project>

eng/pipelines/sdk-perf-jobs.yml

@@ -527,6 +527,82 @@ jobs:
         ${{ each parameter in parameters.jobParameters }}:
           ${{ parameter.key }}: ${{ parameter.value }}
 
+  # Maui Android inner loop benchmarks (Mono Default) - Release
+  - template: /eng/pipelines/templates/build-machine-matrix.yml
+    parameters:
+      jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
+      buildMachines:
+        - win-x64-android-arm64-pixel
+      isPublic: false
+      jobParameters:
+        runKind: maui_scenarios_android_innerloop
+        projectFileName: maui_scenarios_android_innerloop.proj
+        channels:
+          - main
+        runtimeFlavor: mono
+        codeGenType: Default
+        buildConfig: Release
+        additionalJobIdentifier: Mono_InnerLoop
+        ${{ each parameter in parameters.jobParameters }}:
+          ${{ parameter.key }}: ${{ parameter.value }}
+
+  # Maui Android inner loop benchmarks (CoreCLR Default) - Release
+  - template: /eng/pipelines/templates/build-machine-matrix.yml
+    parameters:
+      jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
+      buildMachines:
+        - win-x64-android-arm64-pixel
+      isPublic: false
+      jobParameters:
+        runKind: maui_scenarios_android_innerloop
+        projectFileName: maui_scenarios_android_innerloop.proj
+        channels:
+          - main
+        runtimeFlavor: coreclr
+        codeGenType: Default
+        buildConfig: Release
+        additionalJobIdentifier: CoreCLR_InnerLoop
+        ${{ each parameter in parameters.jobParameters }}:
+          ${{ parameter.key }}: ${{ parameter.value }}
+
+  # Maui Android inner loop benchmarks (Mono Default) - Debug
+  - template: /eng/pipelines/templates/build-machine-matrix.yml
+    parameters:
+      jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
+      buildMachines:
+        - win-x64-android-arm64-pixel
+      isPublic: false
+      jobParameters:
+        runKind: maui_scenarios_android_innerloop
+        projectFileName: maui_scenarios_android_innerloop.proj
+        channels:
+          - main
+        runtimeFlavor: mono
+        codeGenType: Default
+        buildConfig: Debug
+        additionalJobIdentifier: Mono_Debug_InnerLoop
+        ${{ each parameter in parameters.jobParameters }}:
+          ${{ parameter.key }}: ${{ parameter.value }}
+
+  # Maui Android inner loop benchmarks (CoreCLR Default) - Debug
+  - template: /eng/pipelines/templates/build-machine-matrix.yml
+    parameters:
+      jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
+      buildMachines:
+        - win-x64-android-arm64-pixel
+      isPublic: false
+      jobParameters:
+        runKind: maui_scenarios_android_innerloop
+        projectFileName: maui_scenarios_android_innerloop.proj
+        channels:
+          - main
+        runtimeFlavor: coreclr
+        codeGenType: Default
+        buildConfig: Debug
+        additionalJobIdentifier: CoreCLR_Debug_InnerLoop
+        ${{ each parameter in parameters.jobParameters }}:
+          ${{ parameter.key }}: ${{ parameter.value }}
+
   # Maui iOS scenario benchmarks (Mono - Default) - Debug
   - template: /eng/pipelines/templates/build-machine-matrix.yml
     parameters:

src/scenarios/mauiandroidinnerloop/post.py

@@ -0,0 +1,22 @@
+'''
+post cleanup script
+'''
+
+import subprocess
+from performance.logger import setup_loggers, getLogger
+from shared.postcommands import clean_directories
+from test import EXENAME
+
+setup_loggers(True)
+logger = getLogger(__name__)
+
+# Uninstall the app from the connected device so re-runs start from a clean state
+package_name = f'com.companyname.{EXENAME.lower()}'
+logger.info(f"Uninstalling {package_name} from device")
+subprocess.run(['adb', 'uninstall', package_name], check=False)
+
+# Shut down the build server to release file locks before cleanup
+logger.info("Shutting down build server")
+subprocess.run(['dotnet', 'build-server', 'shutdown'], check=False)
+
+clean_directories()

src/scenarios/mauiandroidinnerloop/pre.py

@@ -0,0 +1,77 @@
+'''
+pre-command: Set up a MAUI Android app for deploy measurement.
+Creates the template, restores packages, and prepares the modified file for incremental deploy.
+'''
+import os
+import re
+import sys
+from performance.logger import setup_loggers, getLogger
+from shared import const
+from shared.mauisharedpython import install_latest_maui, MauiNuGetConfigContext
+from shared.precommands import PreCommands
+from test import EXENAME
+
+setup_loggers(True)
+logger = getLogger(__name__)
+logger.info("Starting pre-command for MAUI Android deploy measurement")
+
+precommands = PreCommands()
+
+with MauiNuGetConfigContext(precommands.framework):
+    # Cache NuGet packages locally so they ship to Helix in the workitem payload.
+    # Without this, packages go to the global cache which isn't available on Helix.
+    packages_dir = os.path.join(sys.path[0], '.packages')
+    os.makedirs(packages_dir, exist_ok=True)
+    os.environ['NUGET_PACKAGES'] = packages_dir
+    logger.info(f"Set NUGET_PACKAGES to {packages_dir}")
+
+    install_latest_maui(precommands)
+    precommands.print_dotnet_info()
+
+    precommands.new(template='maui',
+                    output_dir=const.APPDIR,
+                    bin_dir=const.BINDIR,
+                    exename=EXENAME,
+                    working_directory=sys.path[0],
+                    no_restore=False)
+
+    # Fix the .csproj to target only Android (remove iOS, MacCatalyst, Windows TFMs).
+    # The MAUI template targets all platforms, but the Helix machine only has the Android SDK.
+    csproj_path = os.path.join(const.APPDIR, f'{EXENAME}.csproj')
+    with open(csproj_path, 'r') as f:
+        csproj_content = f.read()
+
+    android_tfm = f'{precommands.framework}-android'
+    csproj_content = re.sub(
+        r'<TargetFrameworks>.*?</TargetFrameworks>',
+        f'<TargetFrameworks>{android_tfm}</TargetFrameworks>',
+        csproj_content
+    )
+
+    with open(csproj_path, 'w') as f:
+        f.write(csproj_content)
+
+    logger.info(f"Updated {csproj_path}: TargetFrameworks set to {android_tfm}")
+
+    # Copy the modified MainPage.xaml.cs into src/ for the incremental deploy simulation.
+    # test.py will copy this over app/MainPage.xaml.cs between deploys.
+    src_dir = os.path.join(sys.path[0], const.SRCDIR)
+    os.makedirs(src_dir, exist_ok=True)
+
+    original_file = os.path.join(const.APPDIR, 'MainPage.xaml.cs')
+    modified_file = os.path.join(src_dir, 'MainPage.xaml.cs')
+
+    with open(original_file, 'r') as f:
+        content = f.read()
+
+    # Modify a string literal to trigger assembly recompilation
+    modified_content = content.replace('Hello, World!', 'Hello, World! ')
+
+    if modified_content == content:
+        # Fallback: append a partial class extension to guarantee a code change
+        modified_content = content + '\npartial class MainPage { static string _ts = "modified"; }\n'
+
+    with open(modified_file, 'w') as f:
+        f.write(modified_content)
+
+    logger.info(f"Modified MainPage.xaml.cs written to {modified_file}")

src/scenarios/mauiandroidinnerloop/test.py

@@ -0,0 +1,14 @@
+'''
+MAUI Android Deploy Time Measurement
+Orchestrates first deploy → file edit → incremental deploy → parse binlogs.
+'''
+import os
+from shared.runner import TestTraits, Runner
+
+EXENAME = 'MauiAndroidInnerLoop'
+
+if __name__ == "__main__":
+    traits = TestTraits(exename=EXENAME,
+                        guiapp='false',
+                        )
+    Runner(traits).run()