this post was submitted on 18 Sep 2023
7 points (73.3% liked)

Rust Programming

8161 readers
26 users here now

founded 5 years ago
MODERATORS
 

Is there any library for the queueing mechanism?

What's used by the most - Cron? But a task or rather script executed by Cron won't access to the context of an application. Meaning, a task will have be an independent unit. Whereas I want is a library to use inside a project such that it'll have access to everything.

Anything similar to Sidekiq exist in Rust?

top 17 comments
sorted by: hot top controversial new old
[–] [email protected] 3 points 1 year ago (1 children)

lemmy uses clockwerk i believe, there's a few out there for rust.

[–] [email protected] 6 points 1 year ago* (last edited 1 year ago) (1 children)
let mut scheduler = Scheduler::with_tz(chrono::Utc);

scheduler.every(10.minutes()).plus(30.seconds()).run(|| println!("Periodic task"));

scheduler.every(1.day()).at("3:20 pm").run(|| println!("Daily task")); 

scheduler.every(Tuesday).at("14:20:17").and_every(Thursday).at("15:00").run(|| println!("Biweekly task")); 

Damn, that a really ingenious and intuitive use of the builder pattern.

Kudos to the devs!

[–] [email protected] 0 points 1 year ago (1 children)

Scheduler

what library is it from?

[–] [email protected] 1 points 1 year ago (1 children)
[–] [email protected] 0 points 1 year ago (1 children)

@SuddenlyBlowGreen @nothingness I recently needed that, but fuzzy. Like I want a task ran anywhere between 1PM and 2PM. Can this be accomplished with the clockwerk crate?

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

You could schedule a task at 1 PM, then generate a random number between 0-60 inside that task, wait that many minutes, then launch the actual task.

[–] [email protected] 1 points 1 year ago (1 children)

Can you use async in your project?

If Yes, you can spawn a task that will listen on a channel. If you need to run them in parallel probably you can find a mpmc channel.

If you need for them to run at a specific time, spawn task ,tokio::time::sleep , run job, loop.

Don't know any crate just for this.

[–] [email protected] 0 points 1 year ago

If you need for them to run at a specific time, spawn task ,tokio::time::sleep , run job, loop.

How would you do it every 30 minutes? Every 5 hours? Once a day?

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (1 children)

For my project I just run them. I'm using async so it is just spawning a task in the background, but you can do the same with threads. Don't underestimate the number of threads that you can run on a modern computer.

If you want some sort of throttling you can stuff tasks into a queue and just run N background threads pulling them off an processing them.

If you need durability then you start to have more trouble. This is where I would start looking at a library. IDK if there are any libraries that handle logging, retries and similar, but if not you can probably get the basics down pretty easily.

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago) (1 children)

For my project I just run them.

How would you "just run" a task every 30 minutes? Every 5 hours? Once a day?

[–] [email protected] 2 points 1 year ago (2 children)
std::thread::spawn(|| {
    loop {
        std::thread::sleep(std::time::Duration::from_secs(30*60));
        do_job();
    }
});

Works pretty well. Maybe add a bit of code to crash the whole process on panic or some other logging. Wastes a few KiB of memory per loop but probably not a major issue. Doing this with async will waste only the tiniest amount of memory.

[–] [email protected] 1 points 1 year ago (1 children)

@kevincox How do you stop the job? Do you use channels like in Go?

[–] [email protected] 1 points 1 year ago

It depends. Sometimes you can just put an exit call at the end of main to kill the thread. If you want to attempt graceful shutdown then usually I just use a boolean shutdown flag. Then the loop becomes while !shutdown.get() {

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago) (1 children)

what if one of the calls crashes? how would you re-run it?

[–] [email protected] 1 points 1 year ago (1 children)

Best option is probably to add a wrapper around the thread that re-spawns it. But you can also just catch panics in the loop.

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago)

How would you re-run it multiple times then? An internal should be progressively greater. How would you terminate it if it continues to produce an exception?

[–] [email protected] 1 points 1 year ago

I looked into this last week for recurring tasks in an actix-web project and ended up picking fang, but haven't really battle tested it.. But seems to do what I need. I'm using cron tasks with Postgres as the backend to run a recurring task every minute.