[ad_1]
I have a database server and I fetch data from it. Sometimes data have millions rows and more so I use laziness for downloading. I use Server Side Cursors from clojure.jdbc library https://funcool.github.io/clojure.jdbc/latest/#cursor-queries to fetch data lazily.
Now I have a problem. I need produce initial 500 elements from a lazy-sequence, then the program must wait for 10 minutes to get a signal which report to the program produce next 500 elements and so on until i receive all data from server. But if a report didn`t arrive for 10 minutes, the program must close a connection.
I wrote sample:
(def lazyseq_maps (atom :seq_1 :next_500 false :data nil :seq_2 :next_500 false :data nil)) ; here is a collection of all unfinished lazy sequences waiting for signal to continue produce elements
(jdbc/atomic conn
(with-open [cursor (jdbc/fetch-lazy conn sql]
(let [lazyseq (jdbc/cursor->lazyseq cursor)]
(swap! lazyseq_maps assoc seq_id :next_500 true :data nil)
(loop [lazyseq_rest lazyseq
count 1]
(if (:next_500 (seq_id @lazyseq_maps))
(do
(swap! lazyseq_maps update-in [seq_id :data] conj (first lazyseq_rest))
(when (= 0 (mod count 500))
(swap! lazyseq_maps assoc-in [seq_id :next_500] false))
(recur (rest lazyseq) (inc count)))
;
(func-for-waiting-signal)))) ; here I don`t know how to create function waiting signal to continue fetching data
(seq_id @lazyseq_maps)))
Can you help what clojure tools I should use to solve my problems? I assume I should use core.async to create channels for loops. Am I right? And how should I create function which stop executing loop for 10 minutes or continue executing if I receive appropriate signal?
[ad_2]
لینک منبع
