"We stopped hearing from agent Azure Pipelines 32. Verify the agent machine is running and has a healthy network connection"

[jeffschwMSFT]

Build

https://dev.azure.com/dnceng/internal/_build/results?buildId=2361240

Error:

##[error]We stopped hearing from agent Azure Pipelines 32. Verify the agent machine is running and has a healthy network connection. Anything that terminates an agent process, starves it for CPU, or blocks its network access can cause this error. For more information, see: https://go.microsoft.com/fwlink/?linkid=846610

Build leg reported

No response

Pull Request

No response

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "",
    "BuildRetry": false,
    "ErrorPattern": "We stopped hearing from agent Azure Pipelines .*. Verify the agent machine is running and has a healthy network connection. Anything that terminates an agent process, starves it for CPU, or blocks its network access can cause this error.",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng/internal/_build/results?buildId=2361240
Error message validated: [We stopped hearing from agent Azure Pipelines .*. Verify the agent machine is running and has a healthy network connection. Anything that terminates an agent process, starves it for CPU, or blocks its network access can cause this error.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 2/7/2024 1:03:13 AM UTC

Report

Build	Repository	Step Name	Console log	Pull Request
1602666	dotnet/maui	Controls Layout
1602490	dotnet/maui	Controls SearchBar,Shape,Slider
1602459	dotnet/runtime	osx-arm64 Debug CoreCLR_AllSubsets		dotnet/runtime#134190
1602456	dotnet/runtime	osx-x64 release
1602304	dotnet/maui	Controls Lifecycle,ManualReview,Maps
1602019	dotnet/runtime	iossimulator-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1601713	dotnet/maui	Controls CollectionView6
1601587	dotnet/maui	Controls WebView		dotnet/maui#38601
1601008	dotnet/maui	Controls Cells,CheckBox,ContextActions,CustomRenderers,DatePicker,Dispatcher,DisplayAlert,DisplayPrompt,DragAndDrop
1600800	dotnet/maui	Controls Label
1600783	dotnet/maui	Controls CollectionView4
1600336	dotnet/maui	Controls Editor,Effects,Essentials,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView
1600192	dotnet/runtime	osx-arm64 Debug CoreCLR_AllSubsets		dotnet/runtime#134092
1599757	dotnet/runtime	osx-x64 release CoreCLR_AllSubsets		dotnet/runtime#134078
1598841	dotnet/runtime	osx-arm64 Debug CoreCLR_AllSubsets		dotnet/runtime#133689
1596336	dotnet/runtime	osx-arm64 Release NativeAOT_Libraries		dotnet/runtime#133906
1595488	dotnet/runtime	osx-arm64 Debug CoreCLR_AllSubsets		dotnet/runtime#133597
1593280	dotnet/runtime	osx-arm64 Release NativeAOT		dotnet/runtime#133727
1592602	dotnet/runtime	ios-arm64 Release AllSubsets_CoreCLR_Smoke		dotnet/runtime#133690
1591260	dotnet/runtime	osx-x64 debug Mono_Runtime		dotnet/runtime#133606
1591013	dotnet/runtime	osx-arm64 Release NativeAOT_Libraries		dotnet/runtime#133602
1590893	dotnet/runtime	ios-arm64 Release AllSubsets_NativeAOT_Smoke		dotnet/runtime#133494
1589274	dotnet/runtime	osx-x64 Release AllSubsets_Mono_Interpreter_RuntimeTests monointerpreter	Log	dotnet/runtime#133508
1590019	dotnet/runtime	iossimulator-arm64 Release AllSubsets_Mono
1589955	dotnet/runtime	ios-arm64 Release AllSubsets_NativeAOT_Smoke		dotnet/runtime#133539
1589729	dotnet/runtime	osx-arm64 Release NativeAOT		dotnet/runtime#133529
1589593	dotnet/runtime	iossimulator-x64 Release AllSubsets_Mono
1588495	dotnet/sdk	Mac x64
1587976	dotnet/runtime	ios-arm64 Release AllSubsets_Mono	Log	dotnet/runtime#133445
1586610	dotnet/runtime	osx-x64 Debug Mono_MiniJIT_LibrariesTests		dotnet/runtime#133389
1586625	dotnet/runtime	osx-x64 Debug Mono_MiniJIT_LibrariesTests		dotnet/runtime#133384
1585292	dotnet/maui	Controls Navigation		dotnet/maui#37147
1584662	dotnet/aspnetcore	Test: macOS		dotnet/aspnetcore#69074
1583662	dotnet/runtime	osx-arm64 Release Libraries_Release_CoreCLR
1583537	dotnet/runtime	osx-x64 release CoreCLR_AllSubsets		dotnet/runtime#133093
1583227	dotnet/runtime	osx-arm64 Release AllSubsets_Mono_Interpreter_RuntimeTests monointerpreter
1582510	dotnet/maui	Controls Editor,Effects,Essentials,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView
1582416	dotnet/runtime	osx-x64 Release AllSubsets_Mono_Minijit_RuntimeTests minijit		dotnet/runtime#133223
1582114	dotnet/maui	Controls Shell
1581481	dotnet/runtime	osx-x64 Release NativeAOT		dotnet/runtime#133114
1581456	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_NativeAOT_Smoke		dotnet/runtime#133183
1581228	dotnet/runtime	coreclr Pri0 Runtime Tests Run osx x64 checked
1581111	dotnet/runtime	osx-arm64 release
1580798	dotnet/runtime	osx-arm64 Release CoreCLR_AllSubsets
1580661	dotnet/runtime	osx-arm64 Debug CoreCLR_AllSubsets		dotnet/runtime#133159
1579614	dotnet/maui	Controls Label	Log
1580168	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1579360	dotnet/maui	Controls SearchBar,Shape,Slider		dotnet/maui#38086
1579244	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1579181	dotnet/runtime	osx-arm64 Release NativeAOT		dotnet/runtime#133109
1578056	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1578024	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1577986	dotnet/runtime	osx-x64 Debug Libraries_CheckedCoreCLR		dotnet/runtime#133061
1577520	dotnet/runtime	osx-arm64 Debug Libraries_CheckedCoreCLR		dotnet/runtime#132970
1577318	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_NativeAOT_Smoke		dotnet/runtime#132978
1576744	dotnet/runtime	ios-arm64 Release AllSubsets_CoreCLR_Smoke		dotnet/runtime#133013
1576080	dotnet/runtime	CoreCLR Product Build osx x64 checked		dotnet/runtime#132996
1575659	dotnet/maui	Controls Cells,CheckBox,ContextActions,CustomRenderers,DatePicker,Dispatcher,DisplayAlert,DisplayPrompt,DragAndDrop
1575291	dotnet/maui	Controls Layout		dotnet/maui#37746
1574584	dotnet/maui	Controls Accessibility,ActionSheet,ActivityIndicator,Animation,AppLinks		dotnet/maui#38022
1573930	dotnet/maui	Controls Shell
1573927	dotnet/runtime	osx-arm64 Debug Libraries_CheckedCoreCLR		dotnet/runtime#132934
1573739	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1573706	dotnet/maui	Controls ViewBaseTests,VisualStateManager,Window
1573536	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1572803	dotnet/maui	Controls WebView
1572675	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1572632	dotnet/runtime	osx-arm64 Debug Libraries_CheckedCoreCLR		dotnet/runtime#132797
1572029	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1571553	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1571247	dotnet/runtime	osx-x64 Release Mono_MiniJIT_LibrariesTests
1571086	dotnet/runtime	osx-x64 Release AllSubsets_Mono		dotnet/runtime#132275
1570620	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1569429	dotnet/runtime	coreclr Pri1 Runtime Tests Run osx x64 checked
1569411	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests		dotnet/runtime#132348
1568418	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1567223	dotnet/runtime	ios-arm64 Release AllSubsets_CoreCLR_Smoke		dotnet/runtime#132740
1567122	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1566964	dotnet/maui	Controls ListView
1566262	dotnet/maui	Controls ViewBaseTests,VisualStateManager,Window		dotnet/maui#37438
1565927	dotnet/runtime	ios-arm64 Release AllSubsets_CoreCLR_Smoke		dotnet/runtime#132637
1563837	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1563430	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests	Log
1563425	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1563399	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1563315	dotnet/runtime	osx-arm64 debug Mono_Runtime		dotnet/runtime#132635
1562880	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1562382	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1561753	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests	Log
1561700	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1561283	dotnet/runtime	tvos-arm64 Release AllSubsets_Mono_RuntimeTests	Log
1561204	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_Smoke		dotnet/runtime#131686
1560937	dotnet/runtime	osx-arm64 release CoreCLR_Libraries
1560771	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests
1560207	dotnet/runtime	maccatalyst-arm64 Release AllSubsets_CoreCLR_RuntimeTests

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
7	17	95

[lewing]

#1919 is a similar looking mac host infra failure

[lewing]

as is #1883

[missymessa]

@jeffschwMSFT @lewing is this still an issue?

[lewing]

Yes, there are recent hits

[AraHaan]

This happened again today.