https://vrajat.com/tags/object-storage/Recent content in Object Storage on vrajatHugoenSun, 12 Apr 2026 10:00:00 +0530https://vrajat.com/posts/s3q-s3-backed-queue/Sun, 12 Apr 2026 10:00:00 +0530https://vrajat.com/posts/s3q-s3-backed-queue/<p>This article is an introduction to <a href="https://github.com/vrajat/s3q">s3q</a>, a tool I have been working on. <code>s3q</code> is a queue backed by AWS S3. It stores state in a SQLite database file, caches that file on local disk, and writes the updated file durably to AWS S3 after every durable write.</p> <div class="mermaid-block" data-mermaid-block> <pre class="mermaid-source" hidden>flowchart LR app[Application] subgraph s3q[s3q] direction LR cache[Local SQLite Cache] s3[(Object Storage&lt;br/&gt;S3)] cache --&gt; s3 end app --&gt;|uses| cache</pre> <div class="mermaid-diagram">flowchart LR app[Application] subgraph s3q[s3q] direction LR cache[Local SQLite Cache] s3[(Object Storage&lt;br/&gt;S3)] cache --&gt; s3 end app --&gt;|uses| cache</div> </div><script type="module"> import mermaid from "https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs"; const root = document.documentElement; const blocks = () => document.querySelectorAll("[data-mermaid-block]"); const getMermaidConfig = () => { const styles = getComputedStyle(root); const value = (name, fallback = "") => styles.getPropertyValue(name).trim() || fallback; return { startOnLoad: false, securityLevel: "loose", theme: "base", fontFamily: value("--font-mono", "monospace"), themeVariables: { background: value("--bg-elevated", "#ffffff"), primaryColor: value("--bg-elevated", "#ffffff"), primaryTextColor: value("--text", "#000000"), primaryBorderColor: value("--border-strong", value("--text-muted", "#666666")), lineColor: value("--text-muted", "#666666"), secondaryColor: value("--bg-soft", "#f5f5f5"), tertiaryColor: value("--surface", "#eeeeee"), noteBkgColor: value("--bg-soft", "#f5f5f5"), noteTextColor: value("--text", "#000000"), noteBorderColor: value("--border-strong", value("--text-muted", "#666666")), actorBkg: value("--bg-elevated", "#ffffff"), actorBorder: value("--border-strong", value("--text-muted", "#666666")), actorTextColor: value("--text", "#000000"), signalColor: value("--text", "#000000"), signalTextColor: value("--text", "#000000"), labelBoxBkgColor: value("--bg-soft", "#f5f5f5"), labelBoxBorderColor: value("--border-strong", value("--text-muted", "#666666")), labelTextColor: value("--text", "#000000"), loopTextColor: value("--text", "#000000"), activationBorderColor: value("--accent", "#000000"), activationBkgColor: value("--accent-soft", "#e5e5e5"), sequenceNumberColor: value("--bg", "#ffffff") } }; }; const renderDiagrams = async () => { mermaid.initialize(getMermaidConfig()); for (const block of blocks()) { const source = block.querySelector(".mermaid-source"); const diagram = block.querySelector(".mermaid-diagram"); if (!source || !diagram) { continue; } diagram.removeAttribute("data-processed"); diagram.textContent = source.textContent.trim(); } await mermaid.run({ querySelector: "[data-mermaid-block] .mermaid-diagram" }); }; let frame = null; const scheduleRender = () => { if (frame !== null) { cancelAnimationFrame(frame); } frame = requestAnimationFrame(() => { frame = null; renderDiagrams().catch((error) => { console.error("Failed to render Mermaid diagram", error); }); }); }; scheduleRender(); new MutationObserver((mutations) => { if (mutations.some((mutation) => mutation.attributeName === "data-theme")) { scheduleRender(); } }).observe(root, { attributes: true, attributeFilter: ["data-theme"] }); </script> <p>There are many systems built with object storage as the primary durable layer. Some of them are:</p>