Related blogs :

LTS principle --JobClient Submit task process ( Two )

LTS principle --JobTracker Task receiving and assignment ( Three )

LTS principle --TaskTracker Task processing ( Four )

LTS(light-task-scheduler) It is mainly used to solve the distributed task scheduling problem , Support real-time tasks , Timed tasks and Cron task . It has good flexibility , Expansibility , Robust stability is used by many companies , At the same time, I hope open source fans will contribute together .

Project address

github address :

oschina address :

example :

Document address ( Updating in progress , This shall prevail ):

Both addresses are updated synchronously . be interested , Please add QQ group :109500214 ( Group cipher : hello
world) Discuss together , perfect . More people support it , The more motivated to update , Like to remember the upper right corner star Ha .

##1.7.2-SNAPSHOT(master) Main points of change

* optimization JobContext Medium BizLogger, Removed from the original threadlocal, solve taskTracker The problem of multithreading ,
Remove LtsLoggerFactory.getLogger() usage
Framework overview

LTS There are mainly four kinds of nodes :

* JobClient: Mainly responsible for submitting tasks , And receive feedback results of task execution .
* JobTracker: Responsible for receiving and assigning tasks , task scheduling .
* TaskTracker: Responsible for carrying out tasks , Feedback to JobTracker.
* LTS-Admin:( Management backstage ) Mainly responsible for node management , Task queue management , Monitoring management, etc .
among JobClient,JobTracker,TaskTracker Nodes are stateless . You can deploy multiple and dynamically prune them , To achieve load balancing , Achieve greater load ,
And the framework adopts FailStore Strategy makes LTS It has good fault tolerance .

LTS Registry provides multiple implementations (Zookeeper,redis etc. ), The registration center exposes the node information ,master election .(Mongo or
Mysql) Store task queue and task execution log , netty or mina Communication at the bottom , And provides a variety of serialization methods fastjson, hessian2, java etc. .

LTS Support task type :

* Real time task : Tasks to be performed immediately after submission .
* Timed tasks : Tasks performed at a specified point in time , for example today 3 Point execution ( Single ).
* Cron task :CronExpression, and quartz similar ( But not for use quartz Realized ) for example 0 0/1 * * * ?

Support dynamic modification of task parameters , Setting of task execution time, etc , Support background dynamic adding tasks , support Cron The task is suspended , Supports manual stop of tasks in progress ( Conditional ), Support task monitoring statistics , Support the task execution monitoring of each node ,JVM Monitoring and so on .

Architecture diagram


Concept description

### Node group

* English name NodeGroup, A node group is equivalent to a small cluster , Each node in the same node group is equal , Equivalent , Providing the same services to the outside world .
There is one in each node group master node , this master Nodes are created by LTS Dynamically selected , Be a master After the node is down ,LTS We'll pick another one right away master node , Framework provided API Listen to the user interface .

* seeing the name of a thing one thinks of its function , This is mainly used for failed storage , It is mainly used for node fault tolerance , When remote data interaction fails , Store locally , Waiting for remote communication to resume , Then submit the data .
* FailStore Main users JobClient Task submission for ,TaskTracker Task feedback for ,TaskTracker In the scenario of business log transmission .
FailStore Several implementations are currently available :leveldb,rocksdb,berkeleydb,mapdb,ltsdb, Use to choose which one to use , Users can also use SPI The extension uses its own implementation .
flow chart

The figure below shows a standard real-time task execution process .


LTS-Admin Preview of new version interface

At present, the backstage has ztajy <> Provides a simple authentication function . User name and password in auth.cfg in , User's modification .

## characteristic ###1,Spring support
LTS You can use it completely Spring frame , But considering that it is very useful, it is used in the user project Spring frame , therefore LTS It also provides the right Spring Support for , include Xml And notes , introduce
lts-spring.jar that will do . ###2, Business loggers
stay TaskTracker Business loggers are provided on the client side , For application use , Through this business logger , Business logs can be submitted to JobTracker, These business logs can be accessed through the task ID In series , Can be in LTS-Admin View the execution progress of tasks in real time .
###3,SPI Extended support SPI Expansion can achieve zero invasion , Only the corresponding interface needs to be implemented , It can be realized by LTS use , At present, the open extension interfaces include

* Extension of task queue , Users can choose not to use mysql perhaps mongo Store as queue , You can do it yourself .
* Extension of business logger , Main support at present console,mysql,mongo, Users can also choose to send logs to other places by extension .
###4, Fail over
When the TaskTracker After downtime ,JobTracker It will be allocated to those with downtime immediately TaskTracker All of the tasks are then assigned to other normal tasks TaskTracker Node execution .
###5, Node monitoring
Yes, yes JobTracker,TaskTracker Node for resource monitoring , Task monitoring, etc , Can be real-time in the LTS-Admin Manage background view , And then reasonable resource allocation .
###6, Support of diversified task execution results LTS The framework provides four kinds of implementation results support ,EXECUTE_SUCCESS,EXECUTE_FAILED,EXECUTE_LATER,
EXECUTE_EXCEPTION, And take corresponding processing mechanism for each result , For example, try again .

* EXECUTE_SUCCESS: Successful execution , This situation , Direct feedback client ( If the task is set, feedback to the client ).
* EXECUTE_FAILED: Execution failed , This situation , Direct feedback to client , Do not try again .
EXECUTE_LATER: Later ( Need to try again ), This situation , No feedback to client , Retrying policy adoption 1min,2min,3min Strategy for , The default maximum number of retries is 10 second , Users can modify the number of retries through parameter settings .
* EXECUTE_EXCEPTION: Execution exception , This situation will also be retried ( Retrying policy , ditto )
###7,FailStore fault-tolerant use FailStore Mechanism for node fault tolerance ,Fail And
Store, It will not affect the current application because of the instability of remote communication . specific FailStore explain , Refer to in the concept note FailStore explain .

## Project compilation and packaging The project mainly adopts maven Build , Currently available shell Packaging of scripts . Environmental dependence :Java(jdk1.6+) Maven

There are two types of user use : ###1,Maven structure
It can be done through maven The order will lts Of jar The package is uploaded to the local warehouse . In the father pom.xml Add the corresponding repository, Combined use deploy Upload the command . The specific quotation method can refer to lts The example in .
###2, direct Jar quote Need to lts The modules are packaged separately jar package , And will all lts Dependency package introduction . What are the specific references jar The package can be referred to lts The example in .

##JobTracker and LTS-Admin deploy provide (cmd)windows and (shell)linux There are two versions of scripts to compile and deploy :

Run the sh or build.cmd script , Will be in dist Generated under directory lts-{version}-bin folder

Here is the directory structure , among bin The contents are mainly JobTracker and LTS-Admin Start script for .jobtracker  Yes, it is
JobTracker The configuration file and the jar package ,lts-admin yes LTS-Admin dependent war Packages and configuration files .
lts-{version}-bin File structure of
-- lts-${version}-bin |-- bin | |-- jobtracker.cmd | |-- | |--
lts-admin.cmd | |-- | |-- lts-monitor.cmd | |-- |
|-- |-- conf | |-- | |-- lts-admin.cfg | |--
lts-monitor.cfg | |-- readme.txt | |-- tasktracker.cfg | |-- zoo | |--
jobtracker.cfg | |-- | |-- lts-monitor.cfg |-- lib | |-- *.jar
|-- war |-- jetty | |-- lib | |-- *.jar |-- lts-admin.war
* JobTracker start-up . If you want to start a node , Direct revision conf/zoo Configuration file under , Then run  sh zoo start
that will do , If you want to start two JobTracker node , Then you need to make a copy zoo, For example, it was named zoo2, Revised zoo2 Configuration file under , Then run sh zoo2 start that will do .logs Generated under folder jobtracker-zoo.out journal .
* LTS-Admin start-up . modify conf/lts-monitor.cfg and conf/lts-admin.cfg Configuration under , Then run bin Under the sh or lts-admin.cmd Script is enough .logs Under the folder, the lts-admin.out
journal , If the startup is successful, the access address will be printed in the log , Users can access this address .
##JobClient( deploy ) use Need to introduce lts Of jar Including lts-jobclient-{version}.jar,
lts-core-{version}.jar  And other third party dependence jar. ###API Mode start
JobClient jobClient = new RetryJobClient();
jobClient.setRegistryAddress("zookeeper://"); jobClient.start();
// Submit task Job job = new Job(); job.setTaskId("3213213123");
job.setParam("shopId", "11111");
job.setTaskTrackerNodeGroup("test_trade_TaskTracker"); //
job.setCronExpression("0 0/1 * * * ?"); // support cronExpression expression //
job.setTriggerTime(new Date()); // Support execution at specified time Response response =
###Spring XML Mode start
<bean id="jobClient"
class="com.github.ltsopensource.spring.JobClientFactoryBean"> <property
name="clusterName" value="test_cluster"/> <property name="registryAddress"
value="zookeeper://"/> <property name="nodeGroup"
value="test_jobClient"/> <property name="masterChangeListeners"> <list> <bean
</list> </property> <property name="jobFinishedHandler"> <bean
</property> <property name="configs"> <props> <!-- parameter --> <prop
key="">leveldb</prop> </props> </property> </bean>
###Spring Full annotation mode
@Configuration public class LTSSpringConfig { @Bean(name = "jobClient") public
JobClient getJobClient() throws Exception { JobClientFactoryBean factoryBean =
new JobClientFactoryBean(); factoryBean.setClusterName("test_cluster");
factoryBean.setMasterChangeListeners(new MasterChangeListener[]{ new
MasterChangeListenerImpl() }); Properties configs = new Properties();
configs.setProperty("", "leveldb");
factoryBean.setConfigs(configs); factoryBean.afterPropertiesSet(); return
factoryBean.getObject(); } }
##TaskTracker( Deployment ) Need to introduce lts Of jar Including lts-tasktracker-{version}.jar,
lts-core-{version}.jar  And other third party dependence jar. ### Define your own task execution class
public class MyJobRunner implements JobRunner { @Override public Result
run(JobContext jobContext) throws Throwable { try { // TODO Business logic // Will be sent to LTS
(JobTracker upper ) jobContext.getBizLogger().info(" test , Business log "); } catch
(Exception e) { return new Result(Action.EXECUTE_FAILED, e.getMessage()); }
return new Result(Action.EXECUTE_SUCCESS, " The execution was successful , ha-ha "); } }
###API Mode start
TaskTracker taskTracker = new TaskTracker();
taskTracker.setClusterName("test_cluster"); taskTracker.setWorkThreads(20);
###Spring XML Mode start
<bean id="taskTracker"
init-method="start"> <property name="jobRunnerClass"
value=""/> <property
name="bizLoggerLevel" value="INFO"/> <property name="clusterName"
value="test_cluster"/> <property name="registryAddress"
value="zookeeper://"/> <property name="nodeGroup"
value="test_trade_TaskTracker"/> <property name="workThreads" value="20"/>
<property name="masterChangeListeners"> <list> <bean
</list> </property> <property name="configs"> <props> <prop
key="">leveldb</prop> </props> </property> </bean>
###Spring Start annotation mode
@Configuration public class LTSSpringConfig implements ApplicationContextAware
{ private ApplicationContext applicationContext; @Override public void
setApplicationContext(ApplicationContext applicationContext) throws
BeansException { this.applicationContext = applicationContext; } @Bean(name =
"taskTracker") public TaskTracker getTaskTracker() throws Exception {
TaskTrackerAnnotationFactoryBean factoryBean = new
factoryBean.setMasterChangeListeners(new MasterChangeListener[]{ new
MasterChangeListenerImpl() }); factoryBean.setWorkThreads(20); Properties
configs = new Properties(); configs.setProperty("", "leveldb");
factoryBean.setConfigs(configs); factoryBean.afterPropertiesSet(); //
factoryBean.start(); return factoryBean.getObject(); } }
## Parameter description   Parameter description <>

## Suggestions for use
Usually in a JVM Only one is needed JobClient An example is enough , Do not create a new one for each task JobClient example , This will be a great waste of resources , Because of one JobClient Multiple tasks can be submitted . The same one JVM Generally, try to keep only one TaskTracker An example is enough , More may cause waste of resources . When you meet a TaskTracker When you want to run multiple tasks , Please refer to the following
" One TaskTracker Perform multiple tasks ". ## One TaskTracker Perform multiple tasks
sometimes , Business scenarios require multiple tasks , Some people will ask , Do you want one for each task type TaskTracker To execute . My answer is No , If in a JVM in , It's better to use one TaskTracker To run multiple tasks , Because of one JVM Use multiple in TaskTracker Examples are a waste of resources ( Of course, when you have a lot of tasks , You can use a separate task for this task TaskTracker Node to execute ). So how can we achieve one TaskTracker How about carrying out multiple tasks . The following is a reference example I gave .
/** * Main entrance , stay taskTracker.setJobRunnerClass(JobRunnerDispatcher.class) *
JobClient Submit Task Job type job.setParam("type", "aType") */ public class
JobRunnerDispatcher implements JobRunner { private static final
ConcurrentHashMap<String/*type*/, JobRunner> JOB_RUNNER_MAP = new
ConcurrentHashMap<String, JobRunner>(); static { JOB_RUNNER_MAP.put("aType",
new JobRunnerA()); // You can also Spring Zhongna JOB_RUNNER_MAP.put("bType", new
JobRunnerB()); } @Override public Result run(JobContext jobContext) throws
Throwable { Job job = jobContext.getJob(); String type = job.getParam("type");
return JOB_RUNNER_MAP.get(type).run(job); } } class JobRunnerA implements
JobRunner { @Override public Result run(JobContext jobContext) throws Throwable
{ // TODO A type Job The logic of return null; } } class JobRunnerB implements JobRunner {
@Override public Result run(JobContext jobContext) throws Throwable { // TODO
B type Job The logic of return null; } }
##TaskTracker Of JobRunner test
It's usually under preparation TaskTracker When , Just test JobRunner Is the implementation logic correct , I don't want to start LTS Remote testing . In order to facilitate the test ,LTS Provided JobRunner Quick test method of . Own test class integration
com.github.ltsopensource.tasktracker.runner.JobRunnerTester that will do , And realize initContext and
newJobRunner Method is enough . as lts-examples <>
Examples in :
public class TestJobRunnerTester extends JobRunnerTester { public static void
main(String[] args) throws Throwable { // Mock Job data Job job = new Job();
job.setTaskId("2313213"); JobContext jobContext = new JobContext();
jobContext.setJob(job); JobExtInfo jobExtInfo = new JobExtInfo();
jobExtInfo.setRetry(false); jobContext.setJobExtInfo(jobExtInfo); // Run the test
TestJobRunnerTester tester = new TestJobRunnerTester(); Result result =; System.out.println(JSON.toJSONString(result)); }
@Override protected void initContext() { // TODO initialization Spring container } @Override
protected JobRunner newJobRunner() { return new TestJobRunner(); } }
##Spring Quartz Cron Seamless task access about Quartz Of Cron The task only needs to be in Spring Add code to the configuration to access it LTS platform
<bean class="com.github.ltsopensource.spring.quartz.QuartzLTSProxyBean">
<property name="clusterName" value="test_cluster"/> <property
name="registryAddress" value="zookeeper://"/> <property
name="nodeGroup" value="quartz_test_group"/> </bean>
##Spring Boot support
@SpringBootApplication @EnableJobTracker // start-up JobTracker @EnableJobClient //
start-up JobClient @EnableTaskTracker // start-up TaskTracker @EnableMonitor // start-up Monitor
public class Application { public static void main(String[] args) {, args); } }
All that's left is Add the corresponding configuration in , See lts-example Medium
com.github.ltsopensource.examples.springboot Examples under the package

## Multiple network card selection
When the machine has two internal network cards , occasionally , Users want to make LTS The traffic goes to the external network card , So you need to be in host in , Change the mapping address of host name to the address of external network card , The same in Intranet .

## On node identification
If the node ID is set when the node is started ,LTS One will be set by default UUID Identify the node , The readability will be poor , But it can guarantee the uniqueness of each node , If the user can guarantee the uniqueness of the node ID , It can be done through  
setIdentity  To set the , For example, if each node is deployed on a single machine ( A virtual machine ) upper , Then you can identity Set to host name

##SPI Extended description support JobLogger,JobQueue Wait SPI extend

## Comparison with other solutions

##LTS-Admin use jetty start-up ( default ), Hang up solutions from time to time see issue#389