-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Add E2E degradation testing for DDIL conditions #9586
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 7 commits
da76d26
2bb02fe
6b1d25e
b3b1f1f
e02f7a7
712623c
4b71ddb
61258cd
fc54828
110fcde
4816add
956cb48
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,168 @@ | ||
| name: bandwidth-throttling-linux | ||
| description: Action to throttle the bandwidth on Linux runners (for Android E2E) using tc/netem | ||
|
|
||
| inputs: | ||
| test_server_host: | ||
| description: The host of the test server, no protocol | ||
| required: true | ||
| profile: | ||
| description: "Network profile to use (lte, slow_3g, edge_2g, satellite, flapping). Overrides individual settings." | ||
| required: false | ||
| default: "" | ||
| download_speed: | ||
| description: The download speed limit (in Kbit/s) - used if profile is not set | ||
| required: false | ||
| default: "3300" | ||
| upload_speed: | ||
| description: The upload speed limit (in Kbit/s) - used if profile is not set | ||
| required: false | ||
| default: "3300" | ||
| latency: | ||
| description: The latency (in ms) each way - used if profile is not set | ||
| required: false | ||
| default: "500" | ||
| packet_loss: | ||
| description: Packet loss percentage (0-100) - used if profile is not set | ||
| required: false | ||
| default: "0" | ||
| disable: | ||
| description: Disable throttling | ||
| required: false | ||
| default: "false" | ||
|
|
||
| outputs: | ||
| effective_profile: | ||
| description: The profile that was applied | ||
| value: ${{ steps.resolve-profile.outputs.profile }} | ||
| timeout_multiplier: | ||
| description: Recommended timeout multiplier for this profile | ||
| value: ${{ steps.resolve-profile.outputs.timeout_multiplier }} | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Resolve network profile settings | ||
| id: resolve-profile | ||
| shell: bash | ||
| run: | | ||
| PROFILE="${{ inputs.profile }}" | ||
|
|
||
| # Profile-based settings (same as macOS version) | ||
| case "$PROFILE" in | ||
| lte) | ||
| DOWNLOAD=10000 | ||
| UPLOAD=5000 | ||
| LATENCY=30 | ||
| PACKET_LOSS=0 | ||
| TIMEOUT_MULT=1 | ||
| ;; | ||
| slow_3g) | ||
| DOWNLOAD=400 | ||
| UPLOAD=128 | ||
| LATENCY=300 | ||
| PACKET_LOSS=2 | ||
| TIMEOUT_MULT=3 | ||
| ;; | ||
| edge_2g) | ||
| DOWNLOAD=50 | ||
| UPLOAD=25 | ||
| LATENCY=500 | ||
| PACKET_LOSS=5 | ||
| TIMEOUT_MULT=10 | ||
| ;; | ||
| satellite) | ||
| DOWNLOAD=1000 | ||
| UPLOAD=256 | ||
| LATENCY=700 | ||
| PACKET_LOSS=1 | ||
| TIMEOUT_MULT=5 | ||
| ;; | ||
| flapping) | ||
| DOWNLOAD=1000 | ||
| UPLOAD=256 | ||
| LATENCY=200 | ||
| PACKET_LOSS=0 | ||
| TIMEOUT_MULT=5 | ||
| ;; | ||
| *) | ||
| DOWNLOAD=${{ inputs.download_speed }} | ||
| UPLOAD=${{ inputs.upload_speed }} | ||
| LATENCY=${{ inputs.latency }} | ||
| PACKET_LOSS=${{ inputs.packet_loss }} | ||
| TIMEOUT_MULT=1 | ||
| PROFILE="custom" | ||
| ;; | ||
| esac | ||
|
|
||
| echo "download=$DOWNLOAD" >> $GITHUB_OUTPUT | ||
| echo "upload=$UPLOAD" >> $GITHUB_OUTPUT | ||
| echo "latency=$LATENCY" >> $GITHUB_OUTPUT | ||
| echo "packet_loss=$PACKET_LOSS" >> $GITHUB_OUTPUT | ||
| echo "timeout_multiplier=$TIMEOUT_MULT" >> $GITHUB_OUTPUT | ||
| echo "profile=$PROFILE" >> $GITHUB_OUTPUT | ||
|
|
||
| echo "Network profile: $PROFILE" | ||
| echo " Download: ${DOWNLOAD} Kbit/s" | ||
| echo " Upload: ${UPLOAD} Kbit/s" | ||
| echo " Latency: ${LATENCY} ms" | ||
| echo " Packet Loss: ${PACKET_LOSS}%" | ||
| echo " Timeout Multiplier: ${TIMEOUT_MULT}x" | ||
|
|
||
| - name: Disable existing throttling | ||
| if: ${{ inputs.disable == 'true' }} | ||
| shell: bash | ||
| run: | | ||
| # Remove any existing tc rules | ||
| sudo tc qdisc del dev eth0 root 2>/dev/null || true | ||
| echo "Network throttling disabled" | ||
|
|
||
| - name: Apply network throttling with tc/netem | ||
| if: ${{ inputs.disable != 'true' }} | ||
| shell: bash | ||
| run: | | ||
| # Remove any existing rules | ||
| sudo tc qdisc del dev eth0 root 2>/dev/null || true | ||
|
|
||
| DOWNLOAD="${{ steps.resolve-profile.outputs.download }}" | ||
| LATENCY="${{ steps.resolve-profile.outputs.latency }}" | ||
| PACKET_LOSS="${{ steps.resolve-profile.outputs.packet_loss }}" | ||
|
|
||
| echo "Applying network throttling..." | ||
|
|
||
| # Build netem options | ||
| NETEM_OPTS="delay ${LATENCY}ms" | ||
| if [ "$PACKET_LOSS" != "0" ]; then | ||
| NETEM_OPTS="$NETEM_OPTS loss ${PACKET_LOSS}%" | ||
| fi | ||
|
|
||
| # Add root qdisc with netem for delay and packet loss | ||
| sudo tc qdisc add dev eth0 root handle 1: netem $NETEM_OPTS | ||
|
|
||
| # Add tbf (token bucket filter) for rate limiting | ||
| # burst = rate / 8 (bytes per second / 8 = 1 second of buffer) | ||
| # latency = how long packets can wait in queue | ||
| RATE="${DOWNLOAD}kbit" | ||
| BURST="$((DOWNLOAD / 8))kb" | ||
|
|
||
| sudo tc qdisc add dev eth0 parent 1: handle 2: tbf rate $RATE burst $BURST latency 50ms | ||
|
Comment on lines
+119
to
+147
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🌐 Web query:
💡 Result: Yes.
Practical implication for a client behind you (or a NATed Android emulator):
Sources: Incomplete traffic shaping: The action accepts both 🤖 Prompt for AI Agents |
||
|
|
||
| echo "Network throttling applied:" | ||
| echo " Profile: ${{ steps.resolve-profile.outputs.profile }}" | ||
| sudo tc qdisc show dev eth0 | ||
|
|
||
| - name: Start flapping simulation | ||
| if: ${{ inputs.profile == 'flapping' && inputs.disable != 'true' }} | ||
| shell: bash | ||
| run: | | ||
| # Start the flapping script in background | ||
| nohup ${{ github.action_path }}/flapping-linux.sh & | ||
| FLAPPING_PID=$! | ||
| echo "FLAPPING_PID=$FLAPPING_PID" >> $GITHUB_ENV | ||
| echo "Started flapping simulation with PID: $FLAPPING_PID" | ||
|
|
||
| - name: Test connection after throttling | ||
| if: ${{ inputs.disable != 'true' }} | ||
| shell: bash | ||
| run: | | ||
| echo "Testing connection with throttling applied..." | ||
| curl -o /dev/null -m 30 --retry 2 -s -w 'Total: %{time_total}s\n' 'https://${{ inputs.test_server_host }}/api/v4/system/ping?get_server_status=true' || echo "Connection test completed (may have timed out under heavy throttling)" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| #!/bin/bash | ||
| # Flapping Network Simulation Script for Linux (tc/netem) | ||
| # Simulates intermittent connectivity by cycling through network states | ||
|
|
||
| set -e | ||
|
|
||
| echo "Starting flapping network simulation (Linux)" | ||
| echo "PID: $$" | ||
|
|
||
| # Function to apply network settings using tc/netem | ||
| apply_settings() { | ||
| local download=$1 | ||
| local latency=$2 | ||
| local packet_loss=$3 | ||
| local state_name=$4 | ||
|
|
||
| echo "[$(date '+%H:%M:%S')] Switching to state: $state_name" | ||
|
|
||
| # Remove existing rules | ||
| sudo tc qdisc del dev eth0 root 2>/dev/null || true | ||
|
|
||
| if [ "$state_name" = "disconnected" ]; then | ||
| # 100% packet loss = disconnected | ||
| sudo tc qdisc add dev eth0 root netem loss 100% | ||
| else | ||
| # Build netem options | ||
| NETEM_OPTS="delay ${latency}ms" | ||
| if [ "$packet_loss" != "0" ]; then | ||
| NETEM_OPTS="$NETEM_OPTS loss ${packet_loss}%" | ||
| fi | ||
|
|
||
| # Add netem for delay/loss | ||
| sudo tc qdisc add dev eth0 root handle 1: netem $NETEM_OPTS | ||
|
|
||
| # Add rate limiting | ||
| RATE="${download}kbit" | ||
| BURST="$((download / 8))kb" | ||
| sudo tc qdisc add dev eth0 parent 1: handle 2: tbf rate $RATE burst $BURST latency 50ms | ||
| fi | ||
| } | ||
|
|
||
| # Flapping pattern loop | ||
| cycle=0 | ||
| while true; do | ||
| cycle=$((cycle + 1)) | ||
| echo "[$(date '+%H:%M:%S')] === Flapping cycle $cycle ===" | ||
|
|
||
| # State 1: Connected (good connection) | ||
| apply_settings 1000 200 0 "connected" | ||
| sleep 30 | ||
|
|
||
| # State 2: Disconnected | ||
| apply_settings 0 0 100 "disconnected" | ||
| sleep 5 | ||
|
|
||
| # State 3: Slow 3G | ||
| apply_settings 400 300 2 "slow_3g" | ||
| sleep 30 | ||
|
|
||
| # State 4: Brief disconnection | ||
| apply_settings 0 0 100 "disconnected" | ||
| sleep 3 | ||
|
Comment on lines
+64
to
+78
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. State parameters inconsistent with macOS flapping.sh. The call sites pass only 4 arguments, but comparing to This inconsistency could cause confusion when maintaining both scripts. 🤖 Prompt for AI Agents |
||
| done | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Flapping cleanup is incomplete.
flapping-linux.shis started in the background, but the disable path only deletes the qdisc and never stops that process. A later cleanup can therefore be undone as soon as the flapping loop runs again.Suggested fix
- name: Disable existing throttling if: ${{ inputs.disable == 'true' }} shell: bash run: | + if [ -n "${FLAPPING_PID:-}" ]; then + kill "${FLAPPING_PID}" 2>/dev/null || true + fi + # Remove any existing tc rules sudo tc qdisc del dev eth0 root 2>/dev/null || true echo "Network throttling disabled"Also applies to: 153-160
🤖 Prompt for AI Agents