{ "title": "@upstash/kafka", "description": "A fully typed Kafka client built for Upstash Kafka and HTTP, perfect for serverless and edge runtimes", "date": "2022-01-08T00:00:00.000Z", "url": "https://upstash.com/kafka", "repository": "upstash/upstash-kafka", "body": { "raw": "\n\nAn HTTP/REST based Kafka client built on top of\n[Upstash REST API](https://docs.upstash.com/kafka/rest).\n\nIt is the only connectionless (HTTP based) Kafka client and designed for:\n\n- Serverless functions (AWS Lambda ...)\n- Cloudflare Workers (see the example)\n- Fastly Compute@Edge\n- Next.js Edge, Remix ...\n- Client side web/mobile applications\n- WebAssembly and other environments where HTTP is preferred over TCP\n connections.\n\n# Installation\n\n```bash\nnpm install @upstash/kafka\n```\n\n# Quickstart\n\n## Auth\n\n1. Go to [upstash](https://console.upstash.com/kafka) and select your database.\n2. Copy the `REST API` secrets at the bottom of the page\n\n```typescript\nimport { Kafka } from \"@upstash/kafka\"\n\nconst kafka = new Kafka({\n url: \"\",\n username: \"\",\n password: \"\",\n})\n```\n\n## Produce a single message\n\n```typescript\nconst p = kafka.producer()\nconst message = { hello: \"world\" } // Objects will get serialized using `JSON.stringify`\nconst res = await p.produce(\"\", message)\nconst res = await p.produce(\"\", message, {\n partition: 1,\n timestamp: 12345,\n key: \"\",\n headers: [{ key: \"traceId\", value: \"85a9f12\" }],\n})\n```\n\n## Produce multiple messages.\n\nThe same options from the example above can be set for every message.\n\n```typescript\nconst p = kafka.producer()\nconst res = await p.produceMany([\n {\n topic: \"my.topic\",\n value: { hello: \"world\" },\n // ...options\n },\n {\n topic: \"another.topic\",\n value: \"another message\",\n // ...options\n },\n])\n```\n\n## Consume\n\nThe first time a consumer is created, it needs to figure out the group\ncoordinator by asking the Kafka brokers and joins the consumer group. This\nprocess takes some time to complete. That's why when a consumer instance is\ncreated first time, it may return empty messages until consumer group\ncoordination is completed.\n\n```typescript\nconst c = kafka.consumer()\nconst messages = await c.consume({\n consumerGroupId: \"group_1\",\n instanceId: \"instance_1\",\n topics: [\"test.topic\"],\n autoOffsetReset: \"earliest\",\n})\n```\n\nMore examples can be found in the\n[docstring](https://github.com/upstash/upstash-kafka/blob/main/pkg/consumer.ts#L265)\n\n## Commit manually\n\nWhile `consume` can handle committing automatically, you can also use\n`Consumer.commit` to manually commit.\n\n```typescript\nconst consumerGroupId = \"mygroup\"\nconst instanceId = \"myinstance\"\nconst topic = \"my.topic\"\n\nconst c = kafka.consumer()\nconst messages = await c.consume({\n consumerGroupId,\n instanceId,\n topics: [topic],\n autoCommit: false,\n})\n\nfor (const message of messages) {\n // message handling logic\n\n await c.commit({\n consumerGroupId,\n instanceId,\n offset: {\n topic: message.topic,\n partition: message.partition,\n offset: message.offset,\n },\n })\n}\n```\n\n## Fetch\n\nYou can also manage offsets manually by using `Consumer.fetch`\n\n```typescript\nconst c = kafka.consumer()\nconst messages = await c.fetch({\n topic: \"greeting\",\n partition: 3,\n offset: 42,\n timeout: 1000,\n})\n```\n\n## Examples\n\nSee [/examples](https://github.com/upstash/upstash-kafka/tree/main/examples) as\nwell as various examples in the docstrings of each method.\n\n# Contributing\n\n## Requirements\n\n- [nodejs](https://nodejs.org) v14.x or higher\n- [pnpm](https://pnpm.io/installation)\n\n## Setup\n\n0. Install dependencies using `pnpm install`\n1. Create a kafka instance on upstash.\n [docs](https://docs.upstash.com/kafka#create-a-kafka-cluster)\n2. Create the following topics: `blue`, `red`, `green`.\n [docs](https://docs.upstash.com/kafka#create-a-topic)\n\n The partitions or retention settings don't matter at this time.\n\n3. Create `.env` file with your kafka secrets `cp .env.example .env`\n\n## Running tests\n\n```bash\npnpm test\n```\n", "code": "var Component=(()=>{var h=Object.create;var a=Object.defineProperty;var p=Object.getOwnPropertyDescriptor;var E=Object.getOwnPropertyNames;var y=Object.getPrototypeOf,m=Object.prototype.hasOwnProperty;var u=(n,e)=>()=>(e||n((e={exports:{}}).exports,e),e.exports),F=(n,e)=>{for(var s in e)a(n,s,{get:e[s],enumerable:!0})},o=(n,e,s,r)=>{if(e&&typeof e==\"object\"||typeof e==\"function\")for(let c of E(e))!m.call(n,c)&&c!==s&&a(n,c,{get:()=>e[c],enumerable:!(r=p(e,c))||r.enumerable});return n};var g=(n,e,s)=>(s=n!=null?h(y(n)):{},o(e||!n||!n.__esModule?a(s,\"default\",{value:n,enumerable:!0}):s,n)),f=n=>o(a({},\"__esModule\",{value:!0}),n);var t=u((v,i)=>{i.exports=_jsx_runtime});var b={};F(b,{default:()=>B,frontmatter:()=>N});var l=g(t()),N={title:\"@upstash/kafka\",description:\"A fully typed Kafka client built for Upstash Kafka and HTTP, perfect for serverless and edge runtimes\",date:\"2022-01-08\",url:\"https://upstash.com/kafka\",repository:\"upstash/upstash-kafka\"};function d(n){let e=Object.assign({p:\"p\",a:\"a\",ul:\"ul\",li:\"li\",h1:\"h1\",span:\"span\",div:\"div\",pre:\"pre\",code:\"code\",h2:\"h2\",ol:\"ol\"},n.components);return(0,l.jsxs)(l.Fragment,{children:[(0,l.jsxs)(e.p,{children:[`An HTTP/REST based Kafka client built on top of\n`,(0,l.jsx)(e.a,{href:\"https://docs.upstash.com/kafka/rest\",children:\"Upstash REST API\"}),\".\"]}),`\n`,(0,l.jsx)(e.p,{children:\"It is the only connectionless (HTTP based) Kafka client and designed for:\"}),`\n`,(0,l.jsxs)(e.ul,{children:[`\n`,(0,l.jsx)(e.li,{children:\"Serverless functions (AWS Lambda ...)\"}),`\n`,(0,l.jsx)(e.li,{children:\"Cloudflare Workers (see the example)\"}),`\n`,(0,l.jsx)(e.li,{children:\"Fastly Compute@Edge\"}),`\n`,(0,l.jsx)(e.li,{children:\"Next.js Edge, Remix ...\"}),`\n`,(0,l.jsx)(e.li,{children:\"Client side web/mobile applications\"}),`\n`,(0,l.jsx)(e.li,{children:`WebAssembly and other environments where HTTP is preferred over TCP\nconnections.`}),`\n`]}),`\n`,(0,l.jsxs)(e.h1,{id:\"installation\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#installation\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Installation\"]}),`\n`,(0,l.jsx)(e.div,{\"data-rehype-pretty-code-fragment\":\"\",children:(0,l.jsx)(e.pre,{\"data-language\":\"bash\",\"data-theme\":\"default\",children:(0,l.jsx)(e.code,{\"data-language\":\"bash\",\"data-theme\":\"default\",children:(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"npm\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:\"install\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:\"@upstash/kafka\"})]})})})}),`\n`,(0,l.jsxs)(e.h1,{id:\"quickstart\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#quickstart\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Quickstart\"]}),`\n`,(0,l.jsxs)(e.h2,{id:\"auth\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#auth\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Auth\"]}),`\n`,(0,l.jsxs)(e.ol,{children:[`\n`,(0,l.jsxs)(e.li,{children:[\"Go to \",(0,l.jsx)(e.a,{href:\"https://console.upstash.com/kafka\",children:\"upstash\"}),\" and select your database.\"]}),`\n`,(0,l.jsxs)(e.li,{children:[\"Copy the \",(0,l.jsx)(e.code,{children:\"REST API\"}),\" secrets at the bottom of the page\"]}),`\n`]}),`\n`,(0,l.jsx)(e.div,{\"data-rehype-pretty-code-fragment\":\"\",children:(0,l.jsx)(e.pre,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:(0,l.jsxs)(e.code,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:[(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"import\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" { Kafka } \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"from\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"@upstash/kafka\"'})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:\" \"}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"kafka\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"new\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"Kafka\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"({\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" url: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" username: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" password: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"})\"})})]})})}),`\n`,(0,l.jsxs)(e.h2,{id:\"produce-a-single-message\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#produce-a-single-message\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Produce a single message\"]}),`\n`,(0,l.jsx)(e.div,{\"data-rehype-pretty-code-fragment\":\"\",children:(0,l.jsx)(e.pre,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:(0,l.jsxs)(e.code,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:[(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"p\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" kafka.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"producer\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"()\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"message\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" { hello: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"world\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" } \"}),(0,l.jsx)(e.span,{style:{color:\"#6A737D\"},children:\"// Objects will get serialized using `JSON.stringify`\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"res\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"await\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" p.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"produce\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"(\"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\", message)\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"res\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"await\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" p.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"produce\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"(\"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\", message, {\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" partition: \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"1\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" timestamp: \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"12345\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" key: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" headers: [{ key: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"traceId\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\", value: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"85a9f12\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" }],\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"})\"})})]})})}),`\n`,(0,l.jsxs)(e.h2,{id:\"produce-multiple-messages\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#produce-multiple-messages\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Produce multiple messages.\"]}),`\n`,(0,l.jsx)(e.p,{children:\"The same options from the example above can be set for every message.\"}),`\n`,(0,l.jsx)(e.div,{\"data-rehype-pretty-code-fragment\":\"\",children:(0,l.jsx)(e.pre,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:(0,l.jsxs)(e.code,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:[(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"p\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" kafka.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"producer\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"()\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"res\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"await\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" p.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"produceMany\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"([\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" {\"})}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" topic: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"my.topic\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" value: { hello: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"world\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" },\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#6A737D\"},children:\"// ...options\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" },\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" {\"})}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" topic: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"another.topic\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" value: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"another message\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#6A737D\"},children:\"// ...options\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" },\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"])\"})})]})})}),`\n`,(0,l.jsxs)(e.h2,{id:\"consume\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#consume\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Consume\"]}),`\n`,(0,l.jsx)(e.p,{children:`The first time a consumer is created, it needs to figure out the group\ncoordinator by asking the Kafka brokers and joins the consumer group. This\nprocess takes some time to complete. That's why when a consumer instance is\ncreated first time, it may return empty messages until consumer group\ncoordination is completed.`}),`\n`,(0,l.jsx)(e.div,{\"data-rehype-pretty-code-fragment\":\"\",children:(0,l.jsx)(e.pre,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:(0,l.jsxs)(e.code,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:[(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"c\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" kafka.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"consumer\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"()\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"messages\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"await\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" c.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"consume\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"({\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" consumerGroupId: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"group_1\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" instanceId: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"instance_1\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" topics: [\"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"test.topic\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"],\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" autoOffsetReset: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"earliest\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"})\"})})]})})}),`\n`,(0,l.jsxs)(e.p,{children:[`More examples can be found in the\n`,(0,l.jsx)(e.a,{href:\"https://github.com/upstash/upstash-kafka/blob/main/pkg/consumer.ts#L265\",children:\"docstring\"})]}),`\n`,(0,l.jsxs)(e.h2,{id:\"commit-manually\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#commit-manually\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Commit manually\"]}),`\n`,(0,l.jsxs)(e.p,{children:[\"While \",(0,l.jsx)(e.code,{children:\"consume\"}),` can handle committing automatically, you can also use\n`,(0,l.jsx)(e.code,{children:\"Consumer.commit\"}),\" to manually commit.\"]}),`\n`,(0,l.jsx)(e.div,{\"data-rehype-pretty-code-fragment\":\"\",children:(0,l.jsx)(e.pre,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:(0,l.jsxs)(e.code,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:[(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"consumerGroupId\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"mygroup\"'})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"instanceId\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"myinstance\"'})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"topic\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"my.topic\"'})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:\" \"}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"c\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" kafka.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"consumer\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"()\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"messages\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"await\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" c.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"consume\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"({\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" consumerGroupId,\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" instanceId,\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" topics: [topic],\"})}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" autoCommit: \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"false\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"})\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:\" \"}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"for\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" (\"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"message\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"of\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" messages) {\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#6A737D\"},children:\"// message handling logic\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:\" \"}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"await\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" c.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"commit\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"({\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" consumerGroupId,\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" instanceId,\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" offset: {\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" topic: message.topic,\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" partition: message.partition,\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" offset: message.offset,\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" },\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" })\"})}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"}\"})})]})})}),`\n`,(0,l.jsxs)(e.h2,{id:\"fetch\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#fetch\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Fetch\"]}),`\n`,(0,l.jsxs)(e.p,{children:[\"You can also manage offsets manually by using \",(0,l.jsx)(e.code,{children:\"Consumer.fetch\"})]}),`\n`,(0,l.jsx)(e.div,{\"data-rehype-pretty-code-fragment\":\"\",children:(0,l.jsx)(e.pre,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:(0,l.jsxs)(e.code,{\"data-language\":\"typescript\",\"data-theme\":\"default\",children:[(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"c\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" kafka.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"consumer\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"()\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"const\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"messages\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"=\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#F97583\"},children:\"await\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" c.\"}),(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"fetch\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"({\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" topic: \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:'\"greeting\"'}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" partition: \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"3\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" offset: \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"42\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" timeout: \"}),(0,l.jsx)(e.span,{style:{color:\"#79B8FF\"},children:\"1000\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\",\"})]}),`\n`,(0,l.jsx)(e.span,{className:\"line\",children:(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\"})\"})})]})})}),`\n`,(0,l.jsxs)(e.h2,{id:\"examples\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#examples\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Examples\"]}),`\n`,(0,l.jsxs)(e.p,{children:[\"See \",(0,l.jsx)(e.a,{href:\"https://github.com/upstash/upstash-kafka/tree/main/examples\",children:\"/examples\"}),` as\nwell as various examples in the docstrings of each method.`]}),`\n`,(0,l.jsxs)(e.h1,{id:\"contributing\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#contributing\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Contributing\"]}),`\n`,(0,l.jsxs)(e.h2,{id:\"requirements\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#requirements\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Requirements\"]}),`\n`,(0,l.jsxs)(e.ul,{children:[`\n`,(0,l.jsxs)(e.li,{children:[(0,l.jsx)(e.a,{href:\"https://nodejs.org\",children:\"nodejs\"}),\" v14.x or higher\"]}),`\n`,(0,l.jsx)(e.li,{children:(0,l.jsx)(e.a,{href:\"https://pnpm.io/installation\",children:\"pnpm\"})}),`\n`]}),`\n`,(0,l.jsxs)(e.h2,{id:\"setup\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#setup\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Setup\"]}),`\n`,(0,l.jsxs)(e.ol,{start:\"0\",children:[`\n`,(0,l.jsxs)(e.li,{children:[`\n`,(0,l.jsxs)(e.p,{children:[\"Install dependencies using \",(0,l.jsx)(e.code,{children:\"pnpm install\"})]}),`\n`]}),`\n`,(0,l.jsxs)(e.li,{children:[`\n`,(0,l.jsxs)(e.p,{children:[`Create a kafka instance on upstash.\n`,(0,l.jsx)(e.a,{href:\"https://docs.upstash.com/kafka#create-a-kafka-cluster\",children:\"docs\"})]}),`\n`]}),`\n`,(0,l.jsxs)(e.li,{children:[`\n`,(0,l.jsxs)(e.p,{children:[\"Create the following topics: \",(0,l.jsx)(e.code,{children:\"blue\"}),\", \",(0,l.jsx)(e.code,{children:\"red\"}),\", \",(0,l.jsx)(e.code,{children:\"green\"}),`.\n`,(0,l.jsx)(e.a,{href:\"https://docs.upstash.com/kafka#create-a-topic\",children:\"docs\"})]}),`\n`,(0,l.jsx)(e.p,{children:\"The partitions or retention settings don't matter at this time.\"}),`\n`]}),`\n`,(0,l.jsxs)(e.li,{children:[`\n`,(0,l.jsxs)(e.p,{children:[\"Create \",(0,l.jsx)(e.code,{children:\".env\"}),\" file with your kafka secrets \",(0,l.jsx)(e.code,{children:\"cp .env.example .env\"})]}),`\n`]}),`\n`]}),`\n`,(0,l.jsxs)(e.h2,{id:\"running-tests\",children:[(0,l.jsx)(e.a,{className:\"subheading-anchor\",\"aria-label\":\"Link to section\",href:\"#running-tests\",children:(0,l.jsx)(e.span,{className:\"icon icon-link\"})}),\"Running tests\"]}),`\n`,(0,l.jsx)(e.div,{\"data-rehype-pretty-code-fragment\":\"\",children:(0,l.jsx)(e.pre,{\"data-language\":\"bash\",\"data-theme\":\"default\",children:(0,l.jsx)(e.code,{\"data-language\":\"bash\",\"data-theme\":\"default\",children:(0,l.jsxs)(e.span,{className:\"line\",children:[(0,l.jsx)(e.span,{style:{color:\"#B392F0\"},children:\"pnpm\"}),(0,l.jsx)(e.span,{style:{color:\"#E1E4E8\"},children:\" \"}),(0,l.jsx)(e.span,{style:{color:\"#9ECBFF\"},children:\"test\"})]})})})})]})}function k(n={}){let{wrapper:e}=n.components||{};return e?(0,l.jsx)(e,Object.assign({},n,{children:(0,l.jsx)(d,n)})):d(n)}var B=k;return f(b);})();\n;return Component;" }, "_id": "projects/upstash-kafka.mdx", "_raw": { "sourceFilePath": "projects/upstash-kafka.mdx", "sourceFileName": "upstash-kafka.mdx", "sourceFileDir": "projects", "contentType": "mdx", "flattenedPath": "projects/upstash-kafka" }, "type": "Project", "path": "/projects/upstash-kafka", "slug": "upstash-kafka" }