[00:00:00] Introduction
Alem: observability is everything. For me personally, I don't get something until I can see what's going on visually. That's why I build the dev tools because the dev tools actually show you how like chunks stream in a tool call gets called and approval gets sent,
Andrew: Hello, welcome to Dev Tools fm. This is a podcast about developer tools and the people who make 'em. I'm Andrew, and this is my co-host Justin.
Justin: Uh, so you are the creator of the Tanstack Dev Tools. Is that what you're working on? And the, and working on Tanstack ai. We're really excited to chat about that. Um, but kind of before we dig in, I know that you, uh, have ran, uh, a dev agency in the past. Um, and obviously you're doing a lot of open source. Um, be kind of curious to see how did, how did that happen? How did your career lead you to where you're at today?
Alem: Hey, um, yeah, my name's Alem, and I'm the creator of Tanstack Dev Tools and the co-creator of Tanstack ai. Um, so I actually started my dev agency like three years ago mostly to do consulting and dev work and stuff like that. And yeah, I've been doing software development for like the last decade. And yeah, I decided to try my own thing and I actually joined, uh, G two I like three years ago, and I found a job through them and it was like for a Canadian startup and I had a lot of fun there.
Then I kind of built this cool abstraction on top of react hook form that was specifically built for remix because we were using remix to like build the startup back then. And I just asked like the owner, if it's okay to like open source it, and that's how I kind of spiraled out of control into open source and did all of the stuff I did.
So yeah, that's my backstory on the dev agency side.
Andrew: Yeah, the crazy thought we all have of, oh, I can give away my code for free and then we all just run to do it with all our free time is a, a very odd, odd phenomenon.
Alem: I, uh, I, I, I, I always get a bad look from my wife when I work overtime on open source, but I'm like, yeah, but it's free.
Andrew: Yeah. Yeah. Are you working right now? No, I'm not working, unfortunately.
Alem: Technically it's just.
Andrew: Yeah. Okay. So the, the first big thing that I think you got on most people radars for was the React router dev tools. Uh, it seems like there's a dev tools, uh, theme going on throughout all your projects, but could you fill us in on what the React router dev tools did?
Alem: Yeah, so that, that's kind of a cool story because I always wanted to build like react dev tools and I always, uh, hated the fact that like, the view ecosystem had the best like dev tooling and stuff like that. And I was like always so jealous. And, uh, then after like working on remix, I was like, okay, this is not like a huge project.
Let me try to, well, well, when I say huge, like it's brand new and let me try to like build dev tools for this so I can understand it better and have a lot of fun. And I always wanted to build one of my own and I gave it a shot and like I first built the remix hook form package and then I built like a VS code extension for remix.
And then I was like, okay, it is time for my magnum opus, the dev tools. And that's, that's how I got into it. And I built it. And I'm not gonna lie, it was horrible. Um, I kind of hacked into the internals of remix, which caused a lot of, like, bugs and issues. And I even got in in trouble with Kent because I added it to the Epic stack.
And then the Epic stack had like 10 bugs because of me. And then he threw it, threw it out. So the first versions were horrible, but the later ones were amazing.
Yeah. And, and, and that, that, that's basically my story, be behind React, uh, react dev tools. And they were actually remix development tools before that.
And then IJ
when they moved over to React Travel, I just moved them over. Um, but yeah, it's, it's been hell of a ride. And that's kind of, um, yeah. And, and after building that, I, um.
[00:04:24] Building TanStack Dev Tools
Alem: I was like the, the guy who built the reactor dev tools and then, um, over at Tanstack people were talking about, so they have like, uh, uh, router dev tools. They have query dev tools, and now they wanna bring more dev tools. And how, how, how do you manage that on the screen? Because they had two bubbles already, so you, you don't wanna have 10 of 'em on the screen.
So they wanted to build their dev tools on, um, for it. And Corbin actually reached out to me, um, and to ask me like, Hey, you build the React auto dev tools, wanna help us build Tanstack dev tools? Everyone was like, heck yeah. And that's how I got into building Tanstack dev tools.
Justin: That's a lot of fun. Uh, how has that experience been? Like, have you been able to like build the grand unifying like dev tool here?
Alem: Oh yeah, it's, it's like super amazing because, um, while I was building reactor other dev tools, I made a lot of mistakes, as I already said. I, there were a lot of things that I thought were like, okay, but there was a better way to do them. But I just didn't have the time to like start from scratch, scratch and do it all over again.
And after like they reached out to me, I was like, perfect, I can start from scratch, build the perfect like layer of everything I need to build this. I've already learned what I need and what it should look like and now I can do it properly. And I did. And it's like this super crazy event driven system that like allows you to communicate between the client and the server like easily.
And it's like so many things at once and even I'm even looking into like adding an MCP server that lets you talk with your browser and stuff like that. So it's gonna be crazy.
Andrew: Yeah, I was actually like literally just like two minutes ago thinking like, oh, an MCP server in the dev tools would be amazing. 'cause like if you have all of these dev tools that hook into this common interface for helping you understand your application, that sounds like a treasure trove of information for an LLM when it's trying to debug issues is.
Alem: Yes. And that, that, that's like the, the, the next thing I'm working for dev tools, um, it's, it's kind of close. It's kind of not. I, I just need to figure out like where to plug it and how to make it great for everybody. But yeah, I, I, I need to like talk with Tanner and the rest of the team to like figure it out because they have a lot of other stuff for agents that we are working on that has not yet been really revealed.
So, um, hopefully he won't kill me for saying it here, but yeah, it's gonna be really cool.
[00:07:04] TanStack AI Explained and Differentiators
Justin: Maybe on that note, uh, it'd kind of be cool to talk a little bit about Tanstack ai. I feel like Tenner and co, they're just trying to like build all the things. Uh,
so it was like, especially lately. Yeah. Yeah. So I, I would love to just hear more about like, what is Tanstack ai? Why was it built? Like, you know, where does it land itself in, in terms of other tooling?
Alem: Yeah. Um, so that's a really cool story because. So after I joined, so I, I'm not really a long term Tanstack maintainer. I joined like five months ago or something like that. And I built the dev tools really quickly because I already knew how to do it. And it was almost a matter of like copy pasting the reactor dev tools and just fixing the bad, bad parts.
And, and even like reactor, other dev tools today is just built on top of Tanstack dev tools. So if you're using React other dev tools, you're using Tanstack Dev tools. And after I built that, I built it like two months, I think, uh, before we went into Alpha with it. And this whole time I've been trying to get into AI because I think that's really the, the future of development and I think it's gonna like boost us to incredible heights.
And I was trying to get into it, but I never had anything like concrete to do with ai. I, I don't work like for customers who use AI or something like that. And I don't have like my own successes or anything that run on top of ai. So it was really hard for me to find a way to get into it. And then I thought, okay, maybe I could do Tanstack ai.
And it, it was like a really bold idea and I wasn't sure what's gonna happen and what the reception is gonna look like. And then I wrote it in some of our like private stand stack channels for maintainers. And turns out that Terran already talked with like Jack Harrington and they, they already were talking about like doing it already and it's been like a month in, in, in, in, in the works.
So they just said, yeah, sure. Let, let, let's, uh, let's have you help Jack and do it. And that was like perfect for us because it's allowed us to. Focus on two different, uh, fronts. He was like doing the, the communication between the server and the client and then I was doing the type safety and, and stuff like that.
So we kind and the dev tools as well because AI Tanstack AI has dev tools and yeah, we kind of split in half and did all of this work in like a crazy amount of time. I think we launched Alpha like. After a month of just experimenting. And it's crazy what like AI allows us to do because back then, uh, Opus 4.5 just came out.
And before then it was really hard to use AI in, in open source because AI was like good, but it wasn't like open source level good. And it was usually like create slop and too much code and stuff like that. And then Opus 4.5 just blew my mind. Then I like started using it for Tanstack and, and almost the whole ecosystem was built with Opus 4.5.
It was just basically us, like guard railing it, like directing it and using it and yeah, it's, it's, it's, it's been such a crazy ride and we've built so much with it. I mean, we, we built so much around 10 K already and I'm really excited for the future because we are cooking a lot of stuff. Yeah, it's, it's been a really great journey and it's been amazing.
Like even I, I keep saying this, like I've never built anything that was like really popular. Um, I've built a lot of libraries that grew popularity over time, but it was nothing like overnight. And I think we hit like one K stars in less than a day, which is like mind blowing for me. And yeah, it's just, it's been crazy.
Andrew: You're on two different types of rocket rocket ships right now. One, just AI making things and you riding that wave and then it getting really popular at the same time. So I don't think we've done it, uh, done a, a good job of like actually explaining what Tans Stack AI is and like what it does and how it fits into your workflow.
So can you fill us in on how you might use the package?
Alem: Yeah, for sure. Um, so Tanstack AI is, well, well, for, for the people who already know about like the ISDK, it's just an alternative to the ISDK built from the Tanstack Team. Um, it, it's basically allows you to do. Anything you want with ai. So it's, it's a, it's like a middleman between, uh, your provider and your application.
So it allows you to create agent chats. Um, it allows you to like generate images, generate videos, audio transcriptions, text to speech and stuff like that. And you usually just pick your provider. So the provider is basically open AI and tropic, open router, whatever. Um, you, you pick their, um, uh, adapters is what we call them.
And for example, each, uh, provider has a lot of adapters depending on what they support. So for example, um, OpenAI has a video adapter, image adapter, text adapter, yada yada. And then you just give it to the appropriate method from 10 State K. So for example, if you wanna chat. You say, okay, let me grab the chat method.
Let me grab the open AI adapter, put 'em together. I'm talking with open AI models and it, it works like that. So you wanna generate an image. I grab the image adapter, I grab, um, the generated image method. I put 'em together, I generating images so you can build anything like from simple, like, simple scenarios to something like T three chat.
It's really up to you what you wanna build.
Justin: So there's a lot of folks who are operating in this space who are trying to build these like bridges. And I think they're, they're going after the same sorts of problems, is like you have all the providers and their APIs are different and you kind of want a cohesive interface where you can like switch providers like relatively easily and not get locked, locked into a, a particular provider.
SDKI, I know that like Versel in particular has like a very popular, um, SDK, um, and like I think Tanner had this quote where it's like, you know, there's enough opportunity to improve on like what they're doing that it's like worth like doing your own thing. Can you, can you talk a little bit about how it's like differentiated from some of the other, like clients that exist out there?
Alem: Yeah, I think I'm, I mean, like when you think of. Tanstack you think of type safety. And that's our main differentiator because we really care about your type safety. And you can see it on something like model options. So what we do under the hood is we actually map a model to, uh, the options it accepts.
So basically, for example, let's say you wanna use GPT 5.2. And it supports reasoning. Okay? You can use the model options to send the reasoning config. And for example, you later on figure out, okay, this is really expensive. Let me move to, I dunno, GPT-3 4,5 or something. And, and that model doesn't support reasoning well, you just switch the model.
But we actually tell you like at compile time, hey, this doesn't work, this model doesn't support reasoning. Stuff like that. And, and that's just one of the things, um, we all, we are already working on stuff like, uh, tape type safety and structured outputs and, uh, tools. So, for example, if. Uh, it's really a lot of, um, mix and matching.
So, for example, a lot of open, uh, OpenAI models have support for their, uh, the, the, their own tools like web search and code execution and stuff like that. But for example, I dunno, I, I'm just gonna give random models, but that's not maybe true for them. So for example, if GPT five two supports, um, web search.
Maybe GPT file one doesn't. And then if you so send that tool over, that's not gonna work and stuff like that. So we are also working on actually, uh, telling you at compile time, Hey, this tool does, is not supported, uh, by this model and stuff like that. So that's where we wanna differentiate where. You really don't have to go to the docs.
You really don't have to go Google what each model supports or ask an LLM. It's right there in the code. So as soon as you type the model, you have everything typed for you, and that's it.
Andrew: So with, with that in mind, like having that like nice type safety experience that kind of like, that really limits your options on like API shape and like how the user interacts with your library. So like. You tell us about your journey of like trying to find a good API for all of this that works and satisfies all the constraints you guys are trying to hit.
Alem: Yeah, so that, that's been a hell of a ride. Um, so we, we actually, before we even launched, uh, we were playing with the idea of having, like, um, so for example, if you look at, uh, Elle ISDK, they have, uh, generate text and it's not generate text. Or maybe it's, it is, I can't remember the name, but it's like stream text and generate text, I think.
And then the other ones for structured output. And while looking at that, we wanted to build something that works for like both the object oriented approach and the, the like functionality approach. And we wanted, we were playing around with an API, that's basically like a class that you construct.
Everything. And then you just call stuff like chat and text and stuff like that. Um, but that didn't work for us because we also care about like, uh, bundle splitting and shedding everything that you don't wanna use. And classes are not a great way to do that. So we threw that in the trash and we had a lot of like experiments under the hood that we would build, we were really excited about and then say, okay, this, this doesn't work for X or Y.
And we actually wrote a lot of blog posts on On Tanstack about this, where we actually had this like super cool magical method called ai and it did everything. So basically it, it did image generation, text generation chats, whatever. It did everything. And the only difference would be you just provide a different adapter and then under the hood it, it like adapts the API of, uh, of what you're using to, to, to match what it should be.
And. Sorry. Um, and that was like really cool and it like looked super awesome, um, until you kind of got under the hood of it and you saw like 20, uh, types script overrides, which could never match perfectly. And then we said, okay, let's try it with like inference where you would like infer, okay, if this adapter is inferred with this and it, it extends this, blah, blah, blah, blah, blah.
Let's constrain it like that. And that kind of worked. We then wanted to build composability into it where you could like create the options somewhere and then import them and provide 'em to your chat option. And as soon as we did that, it all fell apart. It was literally impossible to like type and then we said, okay, this is just not worth it.
Let's just be very simple about it. Have at maximum like two or three overloads and that that's it. And that's what we did in the end, and that's why we have. Multiple adapters per functionality plus, uh, multiple methods that allows us to like easily bundle, shake it. It allows us to build better type safety on top, and it all works great.
Andrew: So you only need bundle or tree shaking if your things are running on the client. Uh, what does it mean for this to run on the client versus this to run on the server?
Alem: Yeah, I mean that's, that's a great point. Um, we, we just don't want to give you a code that you're not gonna run anyways.
Andrew: Hmm.
Alem: I know it doesn't matter on the server as much. It's like your, your bundle. Just give it, run it and that's it. So it's not a bigger concern, but we just care about like having it, the minimal footprint possible, at least to be like co friendly.
Andrew: Mm-hmm.
Alem: Um, but yeah, on the client it also matters. Um, but on the client, there are other ways of getting around this and I, I don't think anybody does a bad job at like splitting it on the client. So it's not like, um, I, I think the main differentiator on the client side is not really the, the bundle size, but rather the type safety as well.
Because the way we did tools was we created the definitions and then we have. And then we, we use those definitions on the client to basically give you ultimate type safety on top of the use chat method, where you actually know what tools are available and then, you know, the inputs, outputs and stuff like that.
So it's really easy to use, but I, I, I, I don't think that the bundle size is gonna be the main differentiator there.
Justin: That's cool. Um, I had saw a tweet. Um, on like announcing like AG UI compatibility, um, I don't fully know what that is, but I have heard of like MCP UI and like a lot of like protocols for agents defining, you know, UI components for the front end. And I'm, I'm curious like what that is and sort of like as you're thinking about, um, integration into the broader ecosystem, what is your, what's y'all's strategy been there?
Alem: Yeah, I think that this is kind of our main strategy. So AG UI is an open source protocol made by the team behind copilot kit. And if you know, don't know what that is, it's basically, um. It, it's very similar to to to Tanstack ai, but just a, a little bit more fleshed out. They give you a runtime on the server and they give you client side components that hook into the runtime, and then you can basically stream over events from the server to the client and vice versa.
And it allows you to basically build agent experiences. It's all like event based. So for example, you can subscribe anywhere to anything and like register your own, uh, you, uh, the components in, for example, let's say you render five components. They each have a hook that registers. Some tool call, it all gets like bundled up together and then creates this amazing chat experience.
But that's what copilot is. Copilot kit is, but AG UI is their open, uh, open source, uh, protocol that allows people to standardize, um, communication between the server and the client, and also between agents and between like an agent and the server and. What AG UI allows us to do is basically, let's say, um, you don't wanna use Tanstack AI for the backend.
You want to use something else, Python, PHP, whatever, and then use something like, uh, let's say a lot of, well, AI. And, and let's say a Laravel AI supports AG UI. Well, the cool thing is because we are built on top of a standard, as soon as you start streaming the events from Laravel AI to the client, uh, and let's say, say it.
10 AI is the client. Because the events are standardized, there's no need for you to build like another layer that like translates the events from, uh, layer. Well, AI to our side, it's just standardized and you just listen to the events. So that's it. And that's why we switched to EGUI. Because the, the, the ecosystem is huge and a lot of people are using a GI like Mara Land graph and a lot of other pro, like huge AI projects.
And we just wanna standardize with everybody. And that way we, we, it's easier for people to, uh, like pick, pick the client side, pick the service side, and like pick what they want to use to build the AI agent experience.
Justin: Is this that any does, is there overlap here with Elle's, JSON Render Library? Uh, it seems like there. Might be like a little bit, but I'm not sure how.
[00:24:06] AG UI Standards Explained
Alem: Yeah, so the, this kinds of ties into what I was saying earlier. So, AG UI is an open source protocol for communication between, um, the server and the client. And a, uh, they also have an open source protocol for communication between an agent and an agent. Stuff like that. So there, there are a few protocols.
It's not just one.
And one of those protocols is an alternative to, to j Surrender. And it's also used by Google. So for example, if you want to build a standard on top of the Google standard, you're gonna use the, the one built by copilot kit. If you wanna use only the, the, the, the versa one, you're gonna use Versal.
So that's kind of the difference.
Justin: Gotcha. Makes sense.
Andrew: Confusing naming
Alem: Yeah.
Andrew: put, putting UI in the name and it not being really about UI at all is, uh, an interesting move.
Alem: Yeah. I, I think it's like agent two ui, so it's AG UI at at least. I, I, I, I actually have to check that because Yeah, that, that's actually a good point because, well, I was also confused as to what it is when I first heard it, so I, I feel fully understand your confusion.
Andrew: Yeah, but it's nice that a standard is being adopted throughout all of this so that there's like this, uh, huge, huge bloom of tools and at least they're not all creating their own specifications and they can match up in that like server client, uh, agnostic way that, that seems pretty powerful.
Alem: And, and, and it's like, um, everybody has, everybody has their own different like background and tech stack and stuff like that. And it's like super, super important to standardize because if everybody comes up with their own standards, it's like. Uh, the Wild West and it's like, becomes super hard for you to just use the tools you wanna use, and then you, for example, have to, let's say you want to use the versal ISDK on the server.
Okay? Now I have to. Uh, use their standard and then for example, you want to use Tanstack I. Okay. I have to use their standard and then you have to use land graph. I have to use their standard. And if there's a standard that's actually a standard, then you can just use that and then pick, okay, I want Tanstack for the client.
I want resell AI as the K for the backend. I want another service that uses Land Grab that talks with both of those, and you can just like build whatever you want on top of a standard.
Justin: That's pretty cool. There's also a A two U. I Standard, which is very different and very similar name, which is we just, we just need to improve the, the namings of the standards. The standards are cool. The names could use some love.
Alem: I think A2 UI is the to the J. If I.
Justin: Yeah. More about just like rendering UI from an agent versus like communicating events. Yeah,
Yeah.
yeah. Um, cool. So.
[00:27:10] Observability and Devtools
Justin: One of the hard things about this sort of like agentic world or working with AI tools is just like understanding what's going on, like observability is, is still like a tricky, I know you've done a lot of work on dev tools and, and obviously I'm sure that's carried over into how you're thinking about Tan Stack ai.
So, um, when you're thinking about observability and like how people are tracing what's going on on the system, especially if it's like crossing client server boundaries, like. Are there any like special things that y'all have done to make that easier? Like what's the approach been?
Alem: Yeah, so. The, that actually reminds me of another thing that differentiates Tanstack is that we don't really care what transport mechanism you use. So we, we allow you to basically we, we give you the chunks and you can either send them via as service center events. We are web sockets, we are standard fetch, whatever.
It's really up to you. And I think that's really important because like I said, you don't really know what the tech stack that people are gonna use is. So it's really useful to like, um, not constrain people to use, for example, service center events because if they're, they have some sort of environment constraints where they can't use that, they have to use web sockets, you are already not a viable choice for them.
So I think that's really important for us. And I think, like you said, observability is everything. And that's why we build the dev tools from day one because, um, for me personally, I don't get something until I can see what's going on visually. That's why I build the dev tools because the dev tools actually show you how like chunks stream in and then what's going on and a tool call gets called and approval gets sent, blah, blah, blah.
And I think there's still a ways to go. I'm fully happy with it, where it's like super, super, super transparent and clear what's going on. Because at the moment it's kind of, uh, going for convenience rather than like, uh, details where you just see the main things that you care about, but it's not like nitty gritty in the details.
And I want to get to that point where you can switch between. So you can actually like see every chunk and every detail and what's going on. And I think that's really important and really easy to build with Tanstack Dev tools because I kind of build them for this specific purpose where you just like fire events.
And the observer just listens to them and just does whatever you want. And we have the system in place for it. We just have to build it. And I think that's like the most important thing, especially when you're getting into it because it's such a huge field. It almost feels like, for example, before you would be like a front end specialist, a backend specialist, DevOps, whatever, and this is such a huge field, you can just say like you're an AI specialist, or whatever.
It would make sense because it's like so, so much to grasp and understand what's going on, and it keeps changing every day. It's just like a field you can just go into and specialize and do not do anything else, and you're like, set. So yeah, I think that this is gonna be really important for people who are just coming in and have no idea what's going on because the barrier to entry is huge anyways.
Because if you are either like a frontend specialist, backend specialist, you have to understand the other side. You have to understand the protocols, how it all communicates. How it all communicates with the, with the, the providers and stuff like that. So it's already really hard to get into, and we just wanna make it easier for everybody.
Andrew: Yeah, there's a, a lot of value in being able to see like exactly what the LL M's doing. Like my, especially when it comes to coding tools, if I'm not seeing like what the, the LLM is thinking, uh, that's a loss for me. Uh, it's the same thing with like, I. Skills versus mcps. Like, I like MCPS way more, just 'cause like they're so explicit and like I can see tools being called, I can see inputs and outputs.
I can like follow along. Uh, so yeah, lots and lots of value there.
Alem: Yeah, I agree.
[00:31:14] Adapters Media and Adoption
Justin: Another cool thing that's happened recently is so CloudFlare shipped like a some Tanstack AI support. Um. So you can kind of use it with their workers' AI platform. Uh, but that's like a, that's, that's huge, right? This is like this very, very big service provider, like acknowledging that like, Hey, this thing's cool.
You should use it with our platform. Um, I bet that felt great. Uh, have you seen, uh, other companies starting to, to adopt it? And like what's, you know, what's that been like?
Alem: so, um. It is been crazy because, um, so, so la I think it was last week on Monday, I think, um, Sunil, the, the guy who actually made the support for Tanstack ai, um, he posted something about Vercel AI SDK. On CloudFlare, and then I just said like, Hey, are you gonna do test api support? And he just dd me and said like, oh my God, I love test tech.
I'm gonna do it right now. And he finished it in like two or three days and I was like, wow. And he, yeah, he launched it like last week and, and they were some internal delays that. Launched it on Friday, but he finished it in like two days. Like the man's a legend and it, it's been so exciting, like to see companies excited about test ai and I, I, I gotta say the first company that was really excited and like reached out to us right off the bat to like work with us and launch their adapter.
Was gr and those guys were amazing and they like supported us from day one. So that was really cool for us. And like, the hardest part for us, I, I mean, it's not the hardest part, you just ca cash out the money, but basically like getting access to all these models, um, can be expensive. And it's like super cool.
And an AI company that's excited about the product just says, Hey, here's. I dunno, at least like 20 bucks of tokens. Like go wild, figure out our models and create an adapter. And, and, and like I said, grok has been amazing with this and all Open Router as well, like really great guys, really helped us a lot.
Helped us test their, uh, stuff out, build the adapter for them. And that's like one of the. Biggest ones we have because they support like everything. So if you wanna, I, I, I, I, I really love their product because it really allows me to play around with different models and get a feel of like, what's fast, what's slow, um, what can I use for something?
And it's like really helping me with, um, building Tanstackai because it's, it, if you, if I wanna try, for example, open AI and Anthropic grok, I have to like. Go service by service, buy 10, 20 bucks of tokens and then just keep switching. And this way I can just like buy 10 bucks and then just try all the models and say, okay, this one is good, this one is bad.
I like this one for this. I like this one for that. So that's been a really gr great experience for me as well. And yeah, it's, it's been, uh, great journey. Um, we're seeing a lot of support and it, it's still, it's still in early alpha, so I don't think that companies are excited as they could be. Because it's still like unstable and it's not on the right radar and stuff like that.
So I, I think the future is gonna be a lot brighter than it is right now. But yeah, I, I just can't complain about anything.
Andrew: It makes a lot of sense for these companies, uh, to, to help you guys out. 'cause it's like we just wanna provide models. Uh, you guys can care about the DX in making it a, a nice experience.
Alem: Exactly.
Andrew: Um, so, uh, you guys support a lot of different ways to interact with the models. Uh, does that come with like. Like things that are harder, like there's video generation. Do you have to deal with like streaming the video or do you like care about the ux, like based on the type of media that you're, you're generating?
Or is that kind of all like, like do the models themselves prevent a lot of that from happening?
Alem: Yeah, it's, it's been. Interesting because for example, video generation is something that vercel ai, SDK doesn't support yet. And this was something that we wanted to add in because we really wanted to do video generation because we think it's really cool. And we had a lot of talks about how to do that. And I think, um, we are test tech really, really Tanstack AI really shines is the stuff that's not chat because there's a lot of type safety there.
That's really nuanced and really cool. And for example, uh. When did we launch this? Uh, a few days ago we launched the fal.ai adapter, and they have like over 1000 video and, uh, image models. And, uh, this guy, Tom, from, uh, Australia, really helped us build this and it, it, it basically, uh, constraints. For example, the sizes, the resolutions, the output quality and everything.
So you can't really make a mistake there. For over a thousand models, which is like really cool. And it's like really have difficult to do because, um, usually, uh, every model has its own like constraints on sizes on, uh, the expectation and stuff like that. And I think we really hit it out of the park with this.
And I, I, every time I wanna do some sort of demo with like image generation or video generation, I, I really love using test tag because it's like so simple to do. I don't really have to like, like I said, we bring you the model, all the model knowledge into your code base. So I don't really have to go check what the size is for VO three or whatever.
You are using, uh, VO three as an image model, but you get point, um, and, and that's really cool and you can like really easily use Tanstack for a lot of stuff. So for example. What I've used it for is, uh, so I, I'm also a YouTuber. I'm not sure if anybody knows that, but basically I create a lot of content and, uh, the worst part for me is once I'm done editing and everything, um, I have, I have to go through the video.
Create chapters for YouTube. So for example, for a minute, one 50 chapter, this from 3 28 chapter, this blah, blah, blah. And then I have to like figure out a copy for every social network and then like, it takes me like an hour or two to do. So what I built for myself is I have like a, a docker image of whisper.
Locally, and then I have a script that uses stands and it transcribes. So first of all, I, I grab the audio from the video that I made. I transcribe it, and then I send it over to an LLM. It makes the copies for the socials. It makes me the, the chapters for YouTube. And I've automated everything. So now I just copy paste it into YouTube.
I just like read through it. I correct the parts I don't like, and. Rin and repeat and like there's so many things you can do with tanstack ai I that's like really exciting. It's not like, Hey, I can just build chat interfaces and chat bots. It's like you can do anything you set your mind to and it's really up to you what you wanna do.
Justin: Yeah, that's, that's super awesome. We definitely need something like that for, for our, uh, our production workflow.
Um,
one other thing, uh, Andrew, I think maybe you're getting some feedback. Uh, I don't know. I can't tell. I'm just getting a lot more echoes now than I was previously. Test.
Test. Okay. is it better?
Did your mic change?
Andrew: No,
Justin: ~~No. Okay, All right. All right. cool. I I, I'll restart. Uh, ~~one of the things that I noticed, uh, the like, sort of like last blog post that y'all published about tanstack ai, and I think it's related to the video thing you were just talking about, is, um, y'all broken down the, um, the adapters into, to the separate packages, or is it in the same package still, or is it just like separate modules that they're broken down into?
Alem: It's in the same package. It's just a separate module.
Justin: Great, great. So, so you've broken them, uh, broken those down, which make total sense, right? So it's like if you're using an agent just for chat, um, not paying the weight of, you know, video for example, like all the, uh, configuration and everything goes, that goes with that, like, makes total sense. Um, has that like, unlocked anything for y'all? Is that enabling anything in the future and is like, um, I don't know. When you're thinking about how you build, uh, like. A library like this that seems like it could have such broad, uh, applicability, you know, as like more models add new capabilities. It's like, it could get like pretty big.
Like how do you think about organizing, you know, the capabilities and like, so what makes it into Tanstack ai versus like, oh, I actually want to publish a different package for this.
[00:39:58] Scaling Architecture and Roadmap
Alem: So I think that's the most important part for us, and that that's actually why we broke it up into adapters and, and I'm just gonna explain the story behind it right now. So how it all started. So we are a team of two people. It's me and Jack and we get some help from the community and stuff, but it's mostly the two of us.
And I'm not gonna lie, I'm like overwhelmed. I'm, I'm not just maintaining ai, I'm maintaining dev tools and like 10 remix packages. So I, I just don't have enough time to do everything. And what's really important for us was like to figure out a way to make this scalable. And how we did that was splitting up the adapters.
And why we did that is the following. So. Uh, the initial structure that we launched with the initial alpha had this space adapter that was supposed to have everything, so images, videos, transcription, whatever. And that, that's not like a big problem besides the fact that you get all of that code. But the biggest problem with that was, uh, it had eight generics.
Built in for chat alone. Now imagine adding images, videos, transcription, five different things, translations, whatever comes out in like a year. And that just isn't scalable for us because that first of all means that whenever we want to add, for example, so we've added everything right now because we did this structure, but imagine if we didn't have like images, video, transcription, and whatever, and we now wanted to add images.
We had, uh, providers like OpenAI and Anthropic Grok, um, open Router, whatever else. Let's say there was 30 of them. Now imagine if we added image functionality and now we have to go through 30 adapters and support images or throw in that specific method because it implements the same adapter. It would be like a hell of a thing to do to add a new functionality.
And that's why we decided to split these things up, because now we have like teeny tiny adapters. That's, that's really easy to build. And what this allows us to do is, for example, let's say we wanna support a new, uh, a new provider. Let's say it's grok with Taq. Now what we have to do, let's say they support video, they support images, they support audio, they support whatever.
Um, and let's say we just wanna support text. For now, we can just build a small adapter, launch it, and then for, for example, you care about videos, you are gonna add the video adapter. Uh, somebody else cares about images, they can add the image adapter, and it's like really easy for us to review. Because it's like a teeny tiny pr, it's really easy for somebody to add because it's uh, very tiny.
And that's why we decided to split the adapters up because that's the only way we know how to like, be able to scale this and easily, like merge prs and review them and like get the community support on this because it's not like we have like a. Team of 10 people who are paid to work on this full time.
It's just us working on weekends. So it's really important for us to have this power where we can move forward without like requiring mountains to be moved to get a new feature in. And that's the architecture we aim for.
Andrew: So as we wrap up here, uh, we always like to ask some future facing questions, and it seems like the future is your oyster here. The project is still pretty young and there's still lots to do. So, uh, what can we be on the lookout for from Tans Stack ai.
Alem: Oh yeah. There, there's so many things on the way. I think. So a few things to like, name off the top of my head is like middleware. So we wanna support middleware where you can basically hook into the whole process. So from chunks to responses, to requests to whatever. So that's the first thing we wanna look into.
Then MCP support to allow you to easily build MCP servers. Then we wanna do. there's this cool, cool API that um, we are kind of churning in, in internally and like trying to figure out, um, I, I, I kind of called it, uh, execution workflow because that's kind of what it is. Um, which allows you to like, add retry functionalities, um, use fallback adapters and stuff like that.
So some really advanced use cases, more providers like grok and stuff like that. And yeah, there's, there's so many things we wanna do and so little time. So we have our hands full and every day there's like 20 new things in ai. So it's like, it's very hard to keep, uh, up with everything. So yeah, there, there's ton of work for us to do.
So keep an eye out and just keep following what you're doing.
Justin: That's awesome.
Andrew: thanks for coming on. That wraps up our questions. Uh, this was a really interesting dive into the ai, uh, work you guys are doing.
Uh.
I'm personally gonna integrate it on my project right after this just to see what, what gains I can get. So thanks for coming on and talking about it.
Justin: about it.
Alem: Yeah. Thank you for having me. This was like a really great time and yeah, I, I discovered lately that I love
doing podcasts, so I was really excited to talk with you guys because it feels like,
uh, a talk with your friends and it just get posted online. So it's really cool. And yeah, thanks for having me on.
It was like, pleasure meeting you and talking with you guys.
Justin: Yeah. Love having you. Um. And super excited about what you're working on and, you know, gentle reminder to like, take care of yourself. 'cause open source is hard and open source is especially hard when there's, you know, people using a million AI bots to create issues and file bogus prs and whatever else you're doing.
So thanks for the work. Take care of yourself. Make it sustainable.
Alem: Yeah, that, that, that's always the goal.