Spring Distributed Lock Annotation - Aurora Blog|java/go/python

Overview

Introduce @DistributedLock annotation that can be used together with @Scheduled annotation to execute scheduled task only once in a given time window across all instances of the same Microservice.

Problem

Currently when using @Scheduled annotation for Microservices that have more than one instance, Scheduled Task will be executed multiple times across different Microservices instances at the same time.

In some cases you want to execute Scheduled Task only once across all instances of the same Microservice.

Current Solutions

Solutions exists, however they are not as easy as using single @DistributedLock annotation. Some examples include:

Introduce custom annotation that will use RDBMS as a synchronization point to hold locks across multiple Microservices
Use a Cloud Pattern - for example Create an HTTP function that is triggered by Cloud Scheduler
Use Lambda
Use Event Queue with Single Consumer
Spring Integration Distributed Lock
...

Proposed Solution

Introduce easy to use annotation @DistributedLock that would use some form of abstraction for distributed locks. By default it could use RDBMS.

Usage would look like below:

@DistributedLock(
    lockId = "${payment.check.lock.id}",
    timeout = "${payment.check.lock.timeout}",
    minLockTime = "${payment.check.lock.minLockTime}"
)
@Scheduled(
    fixedDelayString = "${payment.check.task.frequency}",
    initialDelayString = "${payment.check.task.initalDelay}"
)
public void executePaymentCheckTask() {
    ...
}

Details

Min Lock Time

It is important to implement min lock time functionality to prevent execution of the scheduled task multiple times under cases when one instance starts with a slight delay.

Example:

task should be executed every 10 seconds
task execution time is 2 seconds
Instance 2 starts with a 7 seconds delay compared to Instance 1
Because of the delay in startup, task schedule looks like below:
Instance 1: 09:00:00, 09:00:10, 09:00:20, 09:00:30, ...
Instance 2: 09:00:07, 09:00:17, 09:00:27, 09:00:37, ...

Problem: Without min lock time set to 10 seconds, when Instance 1 starts the task at 09:00:00, and when task is finished after 2 seconds at 09:00:02, the Instance 2 would execute the task at 09:00:07.

Solution: Introduce min lock time to prevent the other instance with delayed startup time to execute scheduled task more often then specified by a schedule.

Comment From: mdeinum

The ShedLock library already provides such a feature with many implementations. If you need more control for distributed jobs/scheduling look at libraries like JobRunr or good ol' Quartz.

Comment From: dominikcebula

Would it be possible to consider having ShedLock as an official Spring Project so that it would be covered by Spring Enterprise Support for LTS as well?

Comment From: Sineaggi

Would it be possible to use for example the leader election in spring integration and the spring integration distributed lock to provide support for distributed locking on scheduled jobs?

Comment From: artembilan

I'm not sure how we can handle @Scheduled lifecycle, but in Spring Integration we have a SmartLifecycleRoleController to control if some SmartLifecycle should be run or not in the application. The mentioned Leader Election via distributed locks indeed controls the current application instance state. See more info in docs: https://docs.spring.io/spring-integration/reference/leadership-event-handling.html#page-title

Comment From: Sineaggi

So perhaps instead of a distributed lock annotation, we could register a bean that wraps a task scheduler, and only schedules it if it's the leader? Perhaps via TaskScheduler or SchedulingConfigurer?