Monitor .NET Background Services and Hosted Services with CronPeek API

Your .NET service is running. The health check returns 200. Kubernetes reports all pods healthy. But the nightly invoice generation job that runs inside your BackgroundService stopped executing three days ago after a silent deadlock. No exceptions were thrown, no logs were written, and the process stayed alive. The first sign of trouble is a finance team asking why invoices are missing.

This is the fundamental problem with background job monitoring in .NET. The process can be perfectly healthy while the scheduled work inside it has completely stopped. Traditional uptime monitoring cannot detect this because the failure mode is silence — the absence of something happening.

Dead man's switch monitoring solves this. Your .NET background job pings an external endpoint after every successful run. If the ping stops arriving, you get an alert. This guide shows how to wire that up with IHostedService, BackgroundService, Hangfire, Quartz.NET, ASP.NET Health Checks, and Azure Functions Timer Triggers using CronPeek.

Why .NET Background Jobs Fail Silently

.NET's hosted service model is robust — BackgroundService and IHostedService run inside the same process as your ASP.NET application. But that cohabitation is exactly what masks failures. The web server keeps responding to requests while the background worker is dead. Common silent failure modes in .NET:

• Unobserved task exceptions — an async method throws, but the exception is swallowed because nobody awaits the Task. The ExecuteAsync loop exits, the service stops, no alert fires.
• Thread pool starvation — too many synchronous blocking calls exhaust the thread pool. Your background service's Task.Delay never resumes because no threads are available.
• DbContext lifetime issues — a scoped DbContext injected into a singleton hosted service throws ObjectDisposedException on second use. The service crashes silently on its second iteration.
• Deadlocks — calling .Result or .Wait() on an async method from a synchronization context causes a classic .NET deadlock. The background service freezes, the web host stays alive.
• Configuration drift — a connection string rotates, an API key expires, a certificate is renewed. The background job fails on every attempt but the process keeps running.

None of these are caught by /health endpoints or container probes. You need a monitor that detects when expected work stops happening.

CronPeek Ping Client for .NET

First, build a reusable CronPeekClient that all your background services can use. Register it as a singleton in DI and inject it wherever you need heartbeat pings.

using System;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;

namespace YourApp.Monitoring;

public sealed class CronPeekClient
{
    private const string BaseUrl = "https://cronpeek.web.app/api/v1/ping";
    private readonly HttpClient _http;

    public CronPeekClient(HttpClient http)
    {
        _http = http;
        _http.Timeout = TimeSpan.FromSeconds(5);
    }

    /// <summary>
    /// Sends a heartbeat ping to CronPeek.
    /// Pass succeeded = true for a normal ping, false to report failure.
    /// </summary>
    public async Task PingAsync(
        string monitorId,
        bool succeeded = true,
        CancellationToken ct = default)
    {
        var url = succeeded
            ? $"{BaseUrl}/{monitorId}"
            : $"{BaseUrl}/{monitorId}/fail";

        try
        {
            using var response = await _http.GetAsync(url, ct);
            response.EnsureSuccessStatusCode();
        }
        catch (Exception)
        {
            // Log but never throw — a CronPeek outage
            // must not break your job.
        }
    }
}

Register it in Program.cs:

using YourApp.Monitoring;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHttpClient<CronPeekClient>();

// ... rest of your service configuration

Key details: the 5-second timeout prevents a CronPeek outage from blocking your job. The /fail suffix triggers an immediate alert instead of waiting for a missed heartbeat. Exceptions are swallowed so monitoring never disrupts your business logic.

Monitoring IHostedService and BackgroundService

The BackgroundService base class is the most common way to run periodic work in .NET. Here is a complete example of a timed background service with CronPeek monitoring:

using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using YourApp.Monitoring;

namespace YourApp.Workers;

public sealed class InvoiceGenerationWorker : BackgroundService
{
    private readonly CronPeekClient _cronPeek;
    private readonly IServiceScopeFactory _scopeFactory;
    private readonly ILogger<InvoiceGenerationWorker> _logger;
    private readonly TimeSpan _interval = TimeSpan.FromMinutes(30);

    public InvoiceGenerationWorker(
        CronPeekClient cronPeek,
        IServiceScopeFactory scopeFactory,
        ILogger<InvoiceGenerationWorker> logger)
    {
        _cronPeek = cronPeek;
        _scopeFactory = scopeFactory;
        _logger = logger;
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            var succeeded = false;
            try
            {
                using var scope = _scopeFactory.CreateScope();
                var invoiceService = scope.ServiceProvider
                    .GetRequiredService<IInvoiceService>();

                await invoiceService.GeneratePendingInvoicesAsync(
                    stoppingToken);

                succeeded = true;
                _logger.LogInformation(
                    "Invoice generation completed successfully");
            }
            catch (Exception ex) when (ex is not OperationCanceledException)
            {
                _logger.LogError(ex,
                    "Invoice generation failed");
            }

            // Report to CronPeek after every iteration
            await _cronPeek.PingAsync(
                "mon_invoices_001", succeeded, stoppingToken);

            await Task.Delay(_interval, stoppingToken);
        }
    }
}

Notice the IServiceScopeFactory pattern — this is critical for background services that need scoped dependencies like DbContext. Injecting a scoped service directly into a singleton hosted service is one of the most common causes of silent failures in .NET. Create a CronPeek monitor with an expected interval of 35 minutes (30 minutes plus a 5-minute grace period) to account for job execution time and minor delays.

Monitoring Hangfire Recurring Jobs

Hangfire is the most popular background job library in .NET. For recurring jobs, you can either add CronPeek pings inline or use a global job filter to monitor all jobs automatically.

Inline Monitoring

using Hangfire;
using YourApp.Monitoring;

namespace YourApp.Jobs;

public class ReportGenerationJob
{
    private readonly CronPeekClient _cronPeek;
    private readonly IReportService _reportService;

    public ReportGenerationJob(
        CronPeekClient cronPeek,
        IReportService reportService)
    {
        _cronPeek = cronPeek;
        _reportService = reportService;
    }

    public async Task ExecuteAsync()
    {
        var succeeded = false;
        try
        {
            await _reportService.GenerateDailyReportAsync();
            succeeded = true;
        }
        catch
        {
            throw; // Let Hangfire handle retries
        }
        finally
        {
            await _cronPeek.PingAsync(
                "mon_reports_001", succeeded);
        }
    }
}

// Registration in Startup:
RecurringJob.AddOrUpdate<ReportGenerationJob>(
    "daily-reports",
    job => job.ExecuteAsync(),
    Cron.Daily(2, 0)); // 2:00 AM daily

Global Job Filter

For automatic monitoring across all Hangfire jobs, implement a job filter attribute. This approach avoids modifying every job class:

using System;
using System.Threading.Tasks;
using Hangfire.Common;
using Hangfire.Server;
using YourApp.Monitoring;

namespace YourApp.Filters;

public class CronPeekFilterAttribute : JobFilterAttribute,
    IServerFilter
{
    private readonly string _monitorIdPrefix;

    public CronPeekFilterAttribute(string monitorIdPrefix = "mon_hf")
    {
        _monitorIdPrefix = monitorIdPrefix;
    }

    public void OnPerforming(PerformingContext context)
    {
        // Nothing to do before execution
    }

    public void OnPerformed(PerformedContext context)
    {
        // Build monitor ID from the job name
        var jobName = context.BackgroundJob.Job.Type.Name;
        var monitorId = $"{_monitorIdPrefix}_{jobName}";

        var succeeded = context.Exception == null;

        // Resolve CronPeekClient from the DI container
        var cronPeek = context.Resolve<CronPeekClient>();
        cronPeek.PingAsync(monitorId, succeeded)
            .GetAwaiter().GetResult();
    }
}

// Register globally in Startup:
GlobalJobFilters.Filters.Add(new CronPeekFilterAttribute());

The filter approach means every Hangfire job automatically gets dead man's switch monitoring. Create a CronPeek monitor for each recurring job, naming the monitor ID to match the convention in the filter (mon_hf_ReportGenerationJob).

Monitoring Quartz.NET Scheduled Jobs with IJobListener

Quartz.NET provides a IJobListener interface that hooks into the job lifecycle. This is the cleanest way to add CronPeek monitoring to all Quartz jobs without touching individual job classes.

using System;
using System.Threading;
using System.Threading.Tasks;
using Quartz;
using YourApp.Monitoring;

namespace YourApp.Listeners;

public class CronPeekJobListener : IJobListener
{
    private readonly CronPeekClient _cronPeek;

    public string Name => "CronPeekJobListener";

    public CronPeekJobListener(CronPeekClient cronPeek)
    {
        _cronPeek = cronPeek;
    }

    public Task JobToBeExecuted(
        IJobExecutionContext context,
        CancellationToken ct = default)
    {
        return Task.CompletedTask;
    }

    public Task JobExecutionVetoed(
        IJobExecutionContext context,
        CancellationToken ct = default)
    {
        return Task.CompletedTask;
    }

    public async Task JobWasExecuted(
        IJobExecutionContext context,
        JobExecutionException? jobException,
        CancellationToken ct = default)
    {
        // Derive monitor ID from the job key
        var monitorId = $"mon_qz_{context.JobDetail.Key.Name}";
        var succeeded = jobException == null;

        await _cronPeek.PingAsync(monitorId, succeeded, ct);
    }
}

Register the listener when configuring Quartz in Program.cs:

using Quartz;
using YourApp.Listeners;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddQuartz(q =>
{
    q.AddJobListener<CronPeekJobListener>();

    var jobKey = new JobKey("data-sync");
    q.AddJob<DataSyncJob>(opts => opts.WithIdentity(jobKey));
    q.AddTrigger(opts => opts
        .ForJob(jobKey)
        .WithIdentity("data-sync-trigger")
        .WithCronSchedule("0 0/15 * * * ?"));  // Every 15 minutes
});

builder.Services.AddQuartzHostedService(opts =>
{
    opts.WaitForJobsToComplete = true;
});

Every Quartz job now automatically pings CronPeek with a monitor ID derived from the job key. Create matching monitors in CronPeek — for the example above, create a monitor with ID mon_qz_data-sync and an expected interval of 18 minutes.

Integrating CronPeek with ASP.NET Health Checks

ASP.NET's built-in health check framework integrates with Kubernetes probes, load balancers, and orchestrators. You can bridge CronPeek into this framework so that a failed background job makes the entire health check report degraded.

using System;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Diagnostics.HealthChecks;

namespace YourApp.HealthChecks;

public sealed class CronPeekHealthCheck : IHealthCheck
{
    private static DateTime _lastSuccessfulPing = DateTime.UtcNow;
    private static readonly TimeSpan _maxAge = TimeSpan.FromMinutes(35);

    /// <summary>
    /// Call this from your background service after a successful
    /// CronPeek ping to keep the health check green.
    /// </summary>
    public static void RecordSuccess()
    {
        _lastSuccessfulPing = DateTime.UtcNow;
    }

    public Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken ct = default)
    {
        var age = DateTime.UtcNow - _lastSuccessfulPing;

        if (age < _maxAge)
        {
            return Task.FromResult(
                HealthCheckResult.Healthy(
                    $"Last ping {age.TotalMinutes:F0}m ago"));
        }

        return Task.FromResult(
            HealthCheckResult.Degraded(
                $"No successful ping in {age.TotalMinutes:F0}m"));
    }
}

Register the health check and update your background service to call RecordSuccess():

// Program.cs
builder.Services.AddHealthChecks()
    .AddCheck<CronPeekHealthCheck>(
        "cronpeek-invoice-worker",
        tags: new[] { "background-jobs" });

// In your BackgroundService, after a successful ping:
await _cronPeek.PingAsync("mon_invoices_001", true, stoppingToken);
CronPeekHealthCheck.RecordSuccess();

Now /health will report degraded status if your background job stops running. This gives Kubernetes liveness probes a signal to restart the pod, combining CronPeek's external dead man's switch with the built-in health check infrastructure.

Monitoring Azure Functions Timer Triggers

Azure Functions Timer Triggers are the serverless equivalent of cron jobs in the .NET ecosystem. They can fail silently when the Function App is stopped, a deployment breaks the trigger binding, or the CRON expression is misconfigured.

using System;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Logging;

namespace YourApp.Functions;

public class DataCleanupFunction
{
    private readonly HttpClient _http;
    private readonly ILogger<DataCleanupFunction> _logger;

    public DataCleanupFunction(
        IHttpClientFactory httpFactory,
        ILogger<DataCleanupFunction> logger)
    {
        _http = httpFactory.CreateClient();
        _http.Timeout = TimeSpan.FromSeconds(5);
        _logger = logger;
    }

    [Function("DataCleanup")]
    public async Task RunAsync(
        [TimerTrigger("0 0 3 * * *")] TimerInfo timer)
    {
        var succeeded = false;
        try
        {
            _logger.LogInformation(
                "Data cleanup started at {Time}", DateTime.UtcNow);

            // ... your cleanup logic here
            await CleanupStaleRecordsAsync();

            succeeded = true;
            _logger.LogInformation("Data cleanup completed");
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Data cleanup failed");
            throw; // Let Azure Functions handle retries
        }
        finally
        {
            var endpoint = succeeded
                ? "https://cronpeek.web.app/api/v1/ping/mon_az_cleanup"
                : "https://cronpeek.web.app/api/v1/ping/mon_az_cleanup/fail";

            try
            {
                await _http.GetAsync(endpoint);
            }
            catch
            {
                _logger.LogWarning(
                    "Failed to ping CronPeek — continuing anyway");
            }
        }
    }

    private Task CleanupStaleRecordsAsync()
    {
        // Your actual cleanup implementation
        return Task.CompletedTask;
    }
}

Set the CronPeek monitor's expected interval to 25 hours for a daily job. This gives a 1-hour grace period for Azure's scheduling variance and cold start delays. The finally block ensures the ping is sent even if the job fails, using the /fail endpoint for explicit failure reporting.

For consumption plan functions that may experience cold starts, be generous with your grace period. Cold starts can add 10-30 seconds of latency, and Azure's timer scheduler has its own internal retry logic that may delay execution slightly.

Best Practices: Graceful Shutdown, Retry Logic, and Alert Routing

Graceful Shutdown

When your .NET service shuts down (during deployments, scaling events, or host restarts), the CancellationToken passed to ExecuteAsync is triggered. Make sure your last CronPeek ping happens before the service exits:

protected override async Task ExecuteAsync(
    CancellationToken stoppingToken)
{
    try
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            await DoWorkAsync(stoppingToken);
            await _cronPeek.PingAsync(
                "mon_worker_001", true, stoppingToken);
            await Task.Delay(_interval, stoppingToken);
        }
    }
    catch (OperationCanceledException)
    {
        // Graceful shutdown — expected, not a failure.
        // Set your CronPeek grace period to cover
        // your deploy/restart window.
    }
}

During rolling deployments in Kubernetes, there is a window where no instance is running your background service. Set your CronPeek grace period to cover the maximum deploy time (typically 2-5 minutes for .NET services with readiness probes).

Retry Logic

Your CronPeek pings should be fire-and-forget with a short timeout. Never retry CronPeek pings — if the first attempt fails, the next scheduled job run will send another ping. Retrying adds latency and complexity for no benefit:

• Set HttpClient.Timeout to 5 seconds maximum
• Catch all exceptions and log a warning — never throw
• Do not use Polly or retry policies for CronPeek pings
• If CronPeek is down, your next job iteration will send the ping

Alert Routing

Organize your CronPeek monitors to match your team structure and on-call rotation:

• One monitor per job — never share a monitor ID across different background services. You need to know which job failed.
• Name monitors clearly — use a convention like mon_{service}_{job} so alerts are immediately actionable.
• Route critical jobs to PagerDuty/Slack — payment processing and invoice generation should wake someone up. Report generation can wait until morning.
• Set grace periods generously — if your job runs every 30 minutes, set the expected interval to 35-40 minutes. GC pauses, thread pool contention, and deployment windows all add latency.
• Report failures explicitly — always use the /fail endpoint when you catch an exception. Immediate failure alerts are more useful than waiting for a missed heartbeat timeout.

Monitor Naming Convention

Adopt a consistent naming scheme across all your .NET services:

// BackgroundService workers
mon_invoices_generation
mon_email_queue_processor
mon_data_sync_worker

// Hangfire recurring jobs
mon_hf_DailyReportJob
mon_hf_WeeklyCleanupJob

// Quartz.NET jobs
mon_qz_data-sync
mon_qz_cache-warm

// Azure Functions
mon_az_cleanup
mon_az_billing_export

Consistent naming makes it immediately clear which service and framework a failing monitor belongs to, reducing mean time to resolution when an alert fires at 3 AM.