The Square Developer Podcast

• Building Innovative Mobile Solutions for Restaurants

  Richard Moot: Hello and welcome to the Square Developer Podcast. I'm your host, Richard. Head of Devrel here at Square. And today I'm joined by David and Arielle from Blue Rocket. Thank you so much for being here. Can you go ahead and just give us a quick little intro and tell us about Blue Rocket.David Foote: Blue rocket is a boutique design and development firm, and we kind of cut our teeth on restaurant apps very early in the history of the iPhone. We were the ones that put together Chipotle's first mobile app. And, today, we're still loving restaurants, but we're also spreading out into a lot of AI applications, especially where AI meets the phone.Richard Moot: Very cool. And, what is it that each of you do here at Blue Rocket? Just to set the context for any of our listeners here so we can like, be sure like who's who and who does what.Arielle Watson: So my name is Ariel. I do a little bit of everything. My technical title is VP of Client development. But, in any given week, that might look like working with our development team, working with clients, coding, if I'm lucky, and some design work as well.David Foote: And I'm the CEO at Blue Rocket and haven't always been, but my, my partner retired a couple years ago, and so I stepped up from CTO to CEO, but I'm still I still code on a weekly basis on a daily, daily basis because there's lots of admin and overhead to worry about as well.Richard Moot: I feel like you're describing a little bit of my life. So the majority of like, you know, your expertise within Blue Rocket is like these mobile apps. How is that like your approach to mobile app development sort of evolved over time? Or is it like you came in with, like a certain level of expertise or like, I'd love to know, like a little bit more of like, you know, how you approach those things.David Foote: Well, historically we were very iPhone centric. You know, we've done Android apps along the way, but we've always kind of been more likely to be involved in an iPhone first sort of situation where, where everything was sort of vetted and, and figured out on the iPhone. And then an Android app was created later recently. You know, we tried in like 2016, I think it was we we tried out React Native, and it was just changing so fast that we just couldn't it was just too unstable for us to to do production apps on. But, we tried again recently and we really enjoyed it. It's been a good experience. Retention. So we're actually developing for both Android and iOS at the same time with that.Richard Moot: Excellent. I think you're describing, like, exactly what my feeling has been with React Native for a very long time. And like, there's this very strong love hate relationship where I love it because I'm, I'm mainly a web developer. I know how to build stuff and react. So I felt like, oh, I suddenly feel powerful. I can make web apps or I can make mobile apps.But every time I would come back to an app after, like, I don't know, three, six months, I mean, I'm mostly going to be, like, shocked by my own code. You like who wrote this? But I would also get endlessly frustrated, like, oh, I'm going to go upgrade my dependencies. And oh my gosh, like, I can't get anything to build.And you know, actually X code's on a different version. And it was always a nightmare. So I'm glad to see that there's a little bit more stability here. It feels a little bit more reliable. Have you found that like most of this like expansion with React Native? Do you still do like native Android development or is it still kind of like iOS is like the deeper expertise and then like React Native enables this, like cross-platform, like code reuse?David Foote: We still do Android native development as well. For instance, we're doing some SDK work for a client right now. That's all. It's Java. It's not Kotlin, but it's all Java.Richard Moot: Very cool. And so part of the reason that we wanted to, like, have you on here and chat a little bit is that you've recently for one of your clients, actually started exploring building within Square's ecosystem. I'd love for you to like, tell a little bit more about, like, what brought you into adopting Square and like, how's it going?Arielle Watson: Yeah. So we were approached by a prospective client last year and given what they were out to accomplish, we evaluated a few different vendors. Square was one of them. And for the functionality that we were looking to build, Square had everything that we needed. When we looked at your guys' documentation and looked at the different APIs that you had.And so that was I think that was part of why, from a technical perspective, we recommended going with Square and then, for our client, they had a previous relationship with Square for some other of their businesses. And so they were also leaning that direction. So it worked out really well.Richard Moot: Also. And like, what was it that sort of like, stood out for you in like, sort of meeting the needs of, like, what it is that you were looking for for a platform because, I mean, like, you know, there's I'm going to be honest, there's we know there's a lot of platforms out there, you know. So what is it that sort of stood out and like sort of meeting it? Was it like something within our product sets or like I'd love to know more about what really met the client's needs.Arielle Watson: Yeah. So it was mainly the APIs that you guys offer. So one of the things that they wanted to accomplish was, kind of this dynamic macro calculation as your ordering. And so when we were looking specifically for a way to do that, Square was really the only one that had something that we could leverage for that. So we're using the catalog custom attributes.Richard Moot: And so, can we talk a little bit more about the app in sort of like what it is that you're building. Maybe like we'll roll back a little bit and sort of like you met with a prospective client that like, to what extent you can like to sort of discuss this, like who are they? What were they actually like looking to, to build. And then how did you sort of arrive at, like what you're going to sort of create for that mobile experience?Arielle Watson: Yeah. So bull is a fast casual restaurant that's opening this year in Elk Grove, California. Their offering is to be fast, be customizable to use high quality ingredients. And so when they approached us, being able to appeal to really any user that's on any diet was something they wanted to be able to do. And so the app that we're building supports mobile ordering of course supports loyalty, their rewards.So one of the features that they wanted to support was group ordering. So that a group of people working in an office could just submit one order, make it really simple. And so that's one of the things that we've done. And it was really important to us from a technical perspective, to be able to rely on whichever vendor we chose as the source of truth, so that our backend server could be as lightweight as possible.Richard Moot: Very cool. I feel like you're describing something that's like a dream of mine, my own. I have been really into macro writing for a while. I won't go on like a whole tirade around that, but because I'm, I'm sure nobody actually wants to hear that this isn't like some sort of fitness podcast. But you had previously sort of mentioned Chipotle and like I they're one that like I've seen have like a form of a calculator.I'm not I'm not gonna pretend like this may be the one that you built for them, but like, I know they had like a similar sort of thing of like breaking down calories, like, as you add certain things in, but I think the gap that I've seen in most things is like, I actually want to know, like how much protein, how much fat, how much carbs are in this because I'm basically measuring myself to like, I can only have this much.I love budgeting, and that's probably why I work at Square. I love finance, I love numbers, I'll budget my own food. So it's really cool to, like, be able to leverage something like custom attributes within Catalog. I've been dying to get people to use the custom attributes in this way for this exact reason. I can't tell you the number of developers that I have come in and like, kind of like stuffed stuff into, like different places on the object or put it in somewhere else, or they might just be tracking it on their own somewhere. So it's really great sort of here that you can actually sort of leverage this directly on the catalog and have that kind of nutritional breakdown.Arielle Watson: Yeah, it's been pretty awesome because we're using it for things like macro values on modifiers and on items, but we're also able to utilize it for flags, on those same items for that, that tell the app how it should render something. And so that's been something we didn't necessarily expect to be able to do from the beginning, but it's been really useful.Richard Moot: So one of the things I'm curious about with the app that you're building, because I think, I'm kind of like trying to draw from memory when we sort of talked before the the podcast we're recording here is that there's both a client facing like a customer facing experience, like mobile app. They would have, sort of, on their phone. But there's also like an in-store experience that you guys are also building for them.Arielle Watson: Yeah. Yeah. So we're building the consumer app, which will be on iOS and Android. And then we're also building a kiosk app for them. So that will allow people to just come in and order without needing to log in. And if they want to, they can get their loyalty points.Richard Moot: One of the things I was like, kind of curious about it, like kind of, a little bit change directions or kind of like roll back a little bit, when somebody sort of, like, coming in, like, you have this prospective client and I really kind of want to, like, paint a picture for other folks because, you know, I think I've mostly been in the realm of, like, I've worked for some company and like, I kind of just build software based on, like, their usual process.And I want to sort of understand, like how you approach this because you don't just like, sort of like build apps, like there's a whole experience that you have to sort of walk people through, like what is the actual business need, what are all like the different metrics? I'd love to sort of paint a picture for listeners and like what agency kind of development looks like.So, like, maybe we can start from, like, you have an inbound inquiry come in and you're sort of fielding a request, like what sort of happens from there as you work with the client?David Foote:  So, usually we arrange for a half hour meeting, you know, a video conference meeting just to get a better picture of what their unmet need is what we're trying to accomplish. And, depending on kind of where they're at, what their experience in the software life cycle is, we might recommend that they come in for like a, a free three hour consultation where we, where we kind of build out a more complete definition of the product that they're hoping to build.Or if, if they're very, very early or very new to, to software, we might recommend, a little bit longer of a workshop where they come in and really get, get situated and we can really help them develop, an intuition and a, an overall sense of what it's like to build software. So that might take, you know, we might do a few hours each day for, for three days in a, in a week span in that case. But often we can get somewhere where we're ready to talk about dates and schedules and numbers, in that first three hour workshop.Richard Moot: Okay. So, like in the case of, like this mobile app where you have, like, both an in-person like or, sorry, like, a consumer app and then like sort of an in-store experience, is that something that was like sort of known ahead of time as I was sort of like found through discovery and like trying to figure out, like what it is that they were ultimately solving for.Like, I love to like, sort of understand, like how you sort of like, the best way that I put this, like, when I, like, talk to people in my, my community, is like, sometimes I'm having to sort of, like, pull the, the actual problem out of, like, okay, like, what is it? It's like we're really trying to drive towards here.Like we don't want to have like, really long lines. So you're like, okay, like maybe we want to have a sort of kiosk, like I'm sort of understanding, like was this something that was maybe known ahead of time or is like found through the discovery process?Arielle Watson: Yeah. If I remember, I think Kyle came to us knowing he wanted to do a kiosk app in store.David Foote: And in it definitely, you know, not having or giving people a shot, you know, to avoid the line was definitely a big motivator for that. Because it is an attended the concept is an attended sort of assembly there in the store where they can say, oh, I want some of that, and I want some of that. Like my pizza. I'd like Chipotle. Although my pizza's a bad example. They're kind of disappearing.Richard Moot: I do love my pizza. It is too.David Foote: I like them too, you know.Richard Moot: Because like, I'm always curious. Also, like in the, approach to like user experience between the two of them. And I always have this, like, running theory. I'm maybe validating myself by, like, asking this in different contexts on the podcast. But, one thing I sort of learned was that, like, but it was very different to me, like, say, building an in-person experience versus a consumer experience.Is it in-person? I do feel like a user has maybe like a lower tolerance for waiting on things or load times or like any kind of delays. Whereas like, you know, you would maybe even in a mobile app you might have something slightly more tolerant, like, oh, like it took a little bit like to fetch this data and like rendered in a view.And so I'm just kind of, curious, like what are sort of like the little nuggets of things that you sort of had to sort of change when approaching for, like an in-store experience versus the consumer one.Arielle Watson: I would say we really had to optimize for simplicity and efficiency, because it may be their first time using the kiosk, it may be their first time using any kiosk. And a lot of the kiosk experiences that are out there are really hard to use. So really, we tried to pare down what the user needed to do in order to create an order, even to like where they're entering their name. You know, we just wanted to remove as many roadblocks as possible for them.Richard Moot: And like, are there like sort of in the, in person experience, I guess, like there is like always like a little bit of, like the record of like, oh, I can go because it's intended. So like you could be like, hey, I'm, I'm running into the issue that they can at least have somebody at the store there to hopefully help them in troubleshooting it.If they did run into an issue. Do you have you found like having to implement any kind of like ask for help within there where they can sort of just say like I got stuck or is it kind of like just sort of a they hopefully just back themselves out and start over again.David Foote: We're not open yet. So we haven't had a lot of user feedback for that kind of thing. And I know it's not clear to me yet what the staffing situation would be like for there would be somebody available to come out and help with the kiosk or not. So I guess we'll figure that out.Richard Moot: Yeah. Because I've seen a lot of, like, some other people have built in store experiences. And I mean, I don't want to like, tell like the, the power of, of our platform or tools. I think we actually just get lucky that we have good developers building intuitive apps, on our stuff. But it's more often that like, they've just sort of had a way of resetting back at the beginning again.The only other thing I've sort of heard is from when trying to sort of troubleshoot, I said, like the, a common thing that might that has happened for some folks is, wait a minute, why is mitigating this is like, you know, that for an employee to like, get trained on, like say, how do I reset the the kiosk?That's actually fairly intuitive and not requiring a huge amount of training because, we provide some stuff on like, hey, pear, the reader does this and then it's, it's good to go. But I have heard that like that's like the only other thing that likes tends to need consideration. When, when rolling this out, the only other one I thought was like, accessibility.I wonder, like, is that something that, like, is something that you're thinking about for the initial version of this app or something that's considered like an important requirement? Upfront.Arielle Watson: I would say, one thing that we're kind of always keeping in mind is trying to kind of have multiple cues. So it's not just color, you know, we're trying to use different, different shapes or things like that. So in that sense, I would say it's part of this first version, but anything beyond that, we're definitely interested in.Richard Moot: So for the consumer side of the app, that's mostly to sort of understand like the use case, it's like that's like for like I want to order ahead, maybe curious like is there also going to be a delivery component or is it primarily for like, oh, I want to order online.Arielle Watson: Yeah. So there's going to be a delivery component. And actually the integration with DoorDash, this Square just kind of upgraded was perfect timing for us. Excellent. So yeah, they'll be doing their delivery option, their pickup option. And then they also can save favorites. They can track their loyalty progress. And then the group ordering is another big one.Richard Moot: And then, kind of changing gears a little bit, I'm curious, like, as you're, you know, I know that like, success might be defined as like, hey, you know, we've found, like the right use case, we've landed the client, you've got the work done, we deliver. What does it look like in terms of like, defining success metrics?Like once something is like live and running, and like to what extent do you like, sort of like, continue on in, like maintaining those things? Or is it usually that you would sort of build like an off boarding plan to be like, here's how you can sort of take on maintaining this for your, for you and your, your team going forward.David Foote: So with a, a concept and a company is new is bold. I don't think it would be wise for them to take on the burden of software development, maintenance for some time to come. So, you know, when, when they've gotten big enough that it makes sense to staff that internally, then we'll help them. We'll try to help them transfer that to internal staff. But I wouldn't see that happening for a couple of years yet.Richard Moot: And so like, do you usually sort of. Well, I'm sure it probably works both ways, but do you have a preferred hosting platform that you sort of generally like to build your stuff on, or is it usually just more determined by the client? Hey, I'm sure that some might be more opinionated. Some might be like, yeah, we don't have, we don't really care as long as the backend works.David Foote:  Yeah, usually we're the ones that are recommending, a fairly strong recommendation because we've got, you know, our own routines and reflexes, you know, set up to, to work with certain platforms. Historically, we've been very likely to deploy on Heroku. But more and more, AWS is sort of our default. So in this case, we're using the AWS landers and gateway and probably a couple other things.Richard Moot: Now, one thing I do want to touch on, since you mentioned it, and it's like a fun hot topic and a new personal interest of my own is, you know, working on AI, incorporating AI. I'm just curious, like, how much is it like sort of incorporating or building this for client projects? How much are you actually incorporating it into your own development flows or creative process?David Foote: Well, if they took away ChatGPT tomorrow, we'd be helpless, I feel. Yeah, there, work. Yeah. And we're shifting more and more to Claude for actual coding, for generating code. But, we use ChatGPT for more general questions and, yeah, we use it every day. All day. It makes things possible that before we would have had to, you know, build expertise over several months to tackle, we've got one client that we're doing that we're doing really intricate camera work on, on the iPhone.And that's that's fairly rarefied, you know, that expertise. But, you know, the AI is just helping us just tackle it and not have to be scared. Awesome.Arielle Watson: Yeah. We found it to be really powerful. And as long as you can architect your prompts in a way to get what you need. And then I think David kind of alluded to it earlier, but we've found that, like ChatGPT, has been great for kind of our general questions. But sometimes when we run into, you know, troubleshooting, it can get a little stuck in the loop. And so then the cloud has been really useful for that.Richard Moot: Yeah. I think one tip that I've heard and I'm, I'm, I'm actually starting to like to subscribe to it is that, at least when it comes to coding, it can be very helpful and sort of like building out certain pieces, features, functions, that kind of thing. But I have found that like when you start wrestling with it around debugging, I've had like so much headache and that I've, that's when I've kind of found like, oh, having my software engineering background has been very helpful in these instances of like, oh, if I just go fix the bug and then like, let it keep going, we're going to go ten times faster. If I try to sit here and be like, hey, can you go ahead and fix this bug? It's just going to keep introducing new ones and new ones and new ones and new ones. And then you're just like, okay, this is not. I'm actually taking more time than if I just wrote the function myself.David Foote: You definitely have to wrangle it.Arielle Watson: Yeah. And we found that the more rare the bug is, the less useful, really, any AI has been.Richard Moot: I'm also curious, like for an agency, has it helped in sort of tackling areas where maybe previously you've been like, oh, we don't. It would take us too long to sort of ramp up in this particular area. But at least I'm just curious, like you use it as a tool or to see like, hey, can we maybe figure out like these, like, like what we would think are like hard unknowns to get enough to say yes to like, a certain type of project. I'm just curious, like how is it sort of like unlocked your sort of view of approaching certain types of projects?David Foote: Well, I wouldn't say that. It's made more projects accessible to us. Exactly. So with existing customers where we already sort of have the right to, to work on, certain problems, you know, then we're able to expand how much of that problem we can tackle, maybe replacing some libraries we've relied on before. But, for new customers, it's not enough to give us, you know, bragging rights to say, oh, yeah, we can do that, you know, because we haven't done it yet. We just know that we have good tools to tackle it. If we do, it becomes necessary.Richard Moot: I found like, it's like from a, from a DevRel side of things and just like, sort of exploring things, at least for like when I'm reviewing like code in the language that I'm like, maybe less familiar with, I've even found it, like, helpful to be like, hey, can like you convert this into, like TypeScript or like, explain how this function is working to as if I was a TypeScript developer and it's like suddenly like, oh, okay.Like I see how these different ways that, like you say, rust is working. It's one it's a language that I would love to like, learn but don't have the patience for or, I mean, I probably should have the time, but I definitely realize I don't have the patience for it, so I've definitely found it like very useful for like exploring like these other areas where I would feel like absolutely no confidence to at least be like, well, I'll at least try and see, like what I can come up with.David Foote: That's good to hear. I have a dream of taking one of our Swift UI apps and using AI to generate the Android version of it, so we'll see if that works out.Richard Moot: I would love to know if it does because like, that's like for me, that's like the dream of like, oh, I wrote this example app. I wrote it in TypeScript. Can you go ahead and convert this into Python or Django or some other framework? Like that'd be amazing. And so it would be really, really helpful.David Foote: I'm hoping it's the perfect excuse to spin up cursor AI, because I haven't used that yet, and I understand it has a little wider context across the project than just, single file.Richard Moot: And I mean, I'm no, I'm not supposed to probably be, like, going over here, but I don't really care. That, plug in other tools, but I've definitely tried that one out. And it has been extremely useful. The other one I would recommend that you try out is a shameless plug is that we actually have an AI coding agent that's open source called goose.And it's like it has a CLI and a desktop component. Highly recommend that you try those out. I think all you have to do is you just, like, give it an API key for your line of choice, and it works pretty well. Like you can just sort of say like, hey, I want to like, go do this thing and like, it will actually do what they call like a genetic behavior where it, like, will create files, edit files.You can like, enable all different types of things. I do hope to, like, get, more people even like coming to build on Square to be leveraging that for like even just hammering out your initial proof of concept. I mean, that's the one thing that I found very useful for, like for AI in general, especially for using these agent behaviors.It's not necessarily my default. But sometimes if I have a more complex idea that I want to convey to someone, maybe it involves something like having a small web app or a small website, it would sort of like put everything in a way that would better describe what it is that I'm trying to talk about.It has been really useful for saying, like getting 80, 90% of the way there, where it would probably take me a couple of hours to like, create like a slide deck that would sort of describe this whole thing. And so that part I've actually found to be really handy, just in like a way of using it as a way of expressing the complex thought.Arielle Watson: Yeah, I would agree. I think that being able to take an idea or even a design and turn it into not just an interactive prototype, which like you could do with Figma, but to turn it into like a working POC in a very short period of time. I think that's going to affect our industry hugely in the next few months.Richard Moot: And so one thing I do want to bring this back to Square, just for like kind of coming up towards the end of this in in your experience of sort of building with Square for, you know, these initial like, you know, consumer app in-person experiences, I'd love to know like, and this, you know, be honest with us.What's the good, what's the bad. I'd love to know a little bit more about the overall experience of like, you know, did it make a lot of sense at the beginning. And then you kind of got like a little you stub your toe in places like, I'd love to learn a little bit about, you know, your overall experience with Square and our platform.Arielle Watson: Yeah, I would say overall we've really appreciated the documentation and the API explorer. That's been huge. And I've been leveraging your guys's postman collection a lot more in the last, last month or so. Nice. I think one area that's been more difficult is with using the SDK. Particularly because we are on React Native and we're using Expo, and so there wasn't really out of the box support for that. So it was a little bit of a learning curve to get them working and that we got there.Richard Moot: Don't worry. I'm taking very, very meticulous notes here to remind somebody like you spoke to something that's very close to me. Like I've been wanting them to like have this exposed support. I can say that I've been told that this should be working for mobile payments. SDK should be working with Expo. I'm going to put like a little asterisks in this for the listeners, because I've yet to test this personally.So I have a hard time saying like, yes, this totally works with Expo. But it has been told to me from the people who worked on the React Native, plugin for a mobile payments SDK that it should be working with Expo. I want to say that because like, there's a little bit of caveats when somebody on my team is building an AR tutorial for React Native, which I am preempting, but by the time this comes out, that video will probably be published and you can all watch it and like learn how to use mobile payments SDK with React Native.And then we hope to follow on with another video on how to, how to make this actually work with Expo, specifically because Expo has been like a dream come true when it comes to making your app and your development process so seamless.Arielle Watson: Yeah, kind of go more.Richard Moot: And so I'm sorry for jumping straight in there. I don't know if there was anything else that you wanted to touch on in terms of, overall experience, as you said, Expo. And I got super excited.David Foote: Yeah, yeah.Arielle Watson: No, I think those were the main points for us.Richard Moot: Well, I see that we have come up to our time here. And so I want to thank you both for coming on to the podcast. Big thanks to David, Ariel and all of Blue Rocket, really. And I would love to know if there's anything where folks can go to learn more if they wanted to talk to you about their own mobile app or their own project?Richard Moot: Where can they find you?David Foote: You can find us at www.bluerocket.us.Richard Moot: Awesome. Yeah, yeah. And if you've got a project with Square, we'd love to hear about it. Reach out to us in Discord. Or X at SquareDev. Don't forget to subscribe and check out developer.squareup.com for more. Keep building and catch you next time.

  --------  

  31:06
• Scaling Entertainment Centers and Transforming Guest Experiences

  Richard Moot: Hello and welcome to the Square Developer Podcast. I'm your host, Richard Moody, head of developer relations here at Square. And today I'm joined by Eric and Alex from Headpinz. Eric, tell us a little bit about Headpinz and what it is that you do there.Eric Osborn: Sure. Absolutely. We're a chain of entertainment centers in Southwest Florida. We have everything from bowling to laser tied, the game zones, multiple restaurants, bars, and, soon to be adding an indoor racetrack. The chief information officer for head beans. And then I'm also joined by Alex here, one of our lead developers.Alex Trepasso: I'm Alex. Piggybacking off Eric there with all those different entertainment options and attractions we offer. I pretty much take over integrating them and making our whole ecosystem kind of work as one when we're dealing with all these different systems, you know, from scheduling all the way down to buying a burger, basically integrating those and overseeing the IT operations side of it.Richard Moot: Awesome. And so I don't know if this is quite mentioned, but like how many locations are you guys in the Florida region?Eric Osborn: Right now we're operating two. Well, actually three, two in which our Headpinz, we're actually making a transition from a traditional type bowling centers to more of, a hybrid type environment where we have those leagues on Monday through Friday that people are used to, you know, seeing for the traditional bowlers. But then on late night, Friday night, Saturday, Sunday, we're very much an open bowling venue, open up for families and all that good fun with laser tag and such.Richard Moot: Very cool. And so part of the reason that we're, we're talking today is you have built an integration with Square and it's it's quite interesting, but I'd love to hear the story about, you know, where you using something before. What made you go into using Square. Like tell me a little bit about like, you know what drove you into, you know, using Square through your locations.Eric Osborn: Yeah. So a little bit of a back story. We're actually use Square before our current POS provider decided to do a partnership with Square. We originally started using it for to-go orders during the Covid era when the bowling centers were shut down and we needed to find a way to get our that working.So we were online, and they were doing to-go orders and meeting them right at the door and delivering that food that has since grown into where we've actually made Square our base, then our truth of everything. And that's been a multi-year project. But everything that we do now, including the front end point of sale, our kiosk, our web reservation, anything that is touching financials are now funneling through the Square ecosystem.So a big change over the past three years. And, and that matter of fact, just over the past couple of weeks, as we've finally moved our final processes over to the Square ecosystem. So that's been great. Then where, Alex comes into play and, and, and where the development actually came from was there was some third parties that just simply did not work with Square, such as one of our kiosks, and a couple other small little things like our group function where they actually sign a contract and actually take a deposit.Those things weren't working with Square Alex along and, and worked with, the Square ecosystem on a solution to that. He can speak a little bit about how our kiosks work and such, even though they're not a native, you know, Square partnership that we got to work in on our own.Richard Moot: Yeah, I'd love to like the I mean, that's one of the things that really piqued my interest is you know, you have, well, I don't want to steal thunder here at like, the front of the venue. You have these, like, sort of kiosks where, like, it allows somebody to be able to purchase, you know, various other things, like, other than bowling.And you built this all yourself, essentially. Like, tell me a little bit about, like, the integration that you built with these kiosks and how that all works.Alex Trepasso: Yeah. So it started from a position of when we first got the kiosks, one of their native integrations was another payment provider. And kind of where we came in was, okay, we have these two different systems. You have Square on one side and this other provider on another, both doing, you know, different transaction fee rates, different handling, different view of transactions.And it came up one day in our operations and kind of just our discussions of can we unify these systems. And that led kind of down the rabbit hole of the Square terminal API was kind of our first dive into everything Square developer. And it came along, okay, we know that Square offers this, that we can use this for payments even without sending, you know, a forward or, or doing a whole POS based Square install.Let's try some things from there. We used the terminal API to start listening for those transaction requests that our kiosks were sending to host those on a cloud provider, and it listens for those requests when it sees one, creates a payment request to the terminal and treats it just like any other would for the kiosk. The transaction is completely seamless and paid and just Square it, seeing the payment come through, you know, just as any other transaction, which started kind of our path of Square being our source of truth and opening up the doors of this could be our full ecosystem, even if it's not natively provided by Square, natively provided by our other vendors.Richard Moot: Yeah. And so for the, the kiosk itself, I'm wanting to sort of refresh and also highlight for those, those listening and you know, you primarily do these like a bullion venue and then you have like other things like arcade food and beverage, you're going to be having like the, the go carts. And what all does the kiosk cover in terms of like the guest experience.Eric Osborn: So on your kiosks are we basically covers everything from bowling. We actually have two different types of kiosks, one that is covering bowling and then one that covers all of our other attractions. Those are the ones that you see when you first walk into the facility. From there, you'll actually be able to schedule laser tag ax throwing. You'll be able to buy a Game Zone card, which is, you know, a card like POS system on the game, as well as schedule racing at our other facility.That'll be just across the parking lot that is currently being built. So it's really a one stop shop. Bowling is not handled there because a lot of that does come in through our web payment system. You're booking online ahead of time. These are kind of like those extra attractions when you walk in the door and you're still waiting for your lane or you know you want to find something to do in the interim while you wait for your food.That's kind of what these kiosks address.Richard Moot: I see. And so one of the things you sort of touched on is like, you know, using Square as a source of truth, what kind of like, drove the desire for having that? I mean, and I only ask because, like, you know, I've talked with other people, they might, you know, say they have an ERP system or like, you know, their own bespoke solution.And you know, what kind of started like getting you into the mindset of, of driving towards using Square as your source of truth for, you know, customer data, order data, that kind of stuff.Eric Osborn: Yeah. So we had a plan to, you know, keep moving in the direction of Square wherever possible. However, some things along the way kind of just started to kind of pick up and get aboard the, you know, the path and the ecosystem as we were seeing. It's just easier to funnel the data through Square. That gives us a centralized system to look up all those transactions.It's a centralized system to get all of our financial data over to our online provider. For finances was important to us. And then along the way, we discovered that we were writing all these different tools to make this happen. And we didn't need it. As long as we got the transaction into Square, it was going to do the rest for us.So it really has become that seamless experience. We've taken it a step further and Alex can, you know, mention a couple things about this, but we're even going as far as the Square Catalog is now, that centralized source of truth as well, because it didn't make sense to have to do, you know, before Square came along, I had to put in place keys into three different centers, three different POS centers, or three different POS systems.At each of those centers. And then get it all thinking back together. Whereabouts now, even those third party providers such as, you know, things that do our group function reservations are looking at the Square catalog. Our front desk bowling operation is looking at the Square catalog. So now we have one set of price keys, one set of financials.And you just have that seamless flow into our financial system. And if Alex you want to touch on the thinking because that is something that he custom built as well. For one of the providers to make that happen.Richard Moot: Yeah, I'd love to hear about that because, you know, it's like how often I end up hearing about people trying to sync things between, you know, various different systems. I'd love to learn how you sort of approached it here.Eric Osborn: It's so hard to do it by hand. Though, I'm glad this worked out.Alex Trepasso: Yeah. So it started as kind of a part of our discussion of we're a very report heavy company, and we really like to know what people are doing in our centers. We like being able to have an idea of what's going on. So it started with, okay, our reports at this point were just showing, you know, here's a customer amount transaction coming from a kiosk or this kind of uncategorized revenue that we we knew was coming in certain ways, but we didn't know what people were actually buying.So we started down that path and it came upon a point of, okay, we're managing three different centers of systems here. We're managing three different locations of different products. What can we do to centralize? This started with our group functions, especially because they were a really good example of, you know, you have hundreds of different price keys that each center might use, different pricing and all that.And we began down the path of creating a system to actually synchronize the Square catalog to our group function booking system that my, my thinking there kind of started with. Okay, so Square organizes items by category and it can have these different prices, different availability locations. Whereas our group function system really only has like, you know, one category layer kind of it has the ability to do this price keys, but not as in-depth as Square allows.So we started with the idea of every location that's available. We're actually going to treat it as an independent price key on the vendor side. Whereas the Square it's one we're actually going to create three prices. So if we create a group function price for, you know, an hour of bowling on the Square side, we're seeing, you know, that uniform price, the price over it, any overrides for the location where it's available.And then I'm turning around and taking that information, including the category, and it's kind of the setup of it and creating three separate keys off of that and saying, hey, all three of these are related back to this one in Square. But that vendor system doesn't need to know that that synchronization handles all of that actual communication. So if we update the price for one location, it will only update that singular version on the vendor side.But if we want to update all three at once in Square, that would just be one little change. And then the system would synchronize those changes over to the outside system. The biggest hurdle that we kind of ran into, is Square keeps everything very clean. And its catalog, it is very kind of intuitive. A lot of these outside systems have these layers of complexity that we didn't feel for.At least our setup needs to exist. And Eric could explain more on that of just, you know, we don't need to go through five pages to find one price key where Square has it right ready to go. And we can set up kind of how intuitively a customer or a staff member would actually want to look at that.Eric Osborn: And that now reminds me, Richard, the thumb before let us into a whole nother web of what we could do marketing wise as we were. You know, we originally only just using Square for food and beverage, but then as it grew into our other systems that grew into bowling and laser tag and the game zone, all of a sudden our marketing capabilities, the doors opened wide open.We can now see that, you know, when a guest walks in the door, are they simply coming in for bowling or they coming in for bowling and and buying a couple drinks and buying food by and laser tag. We can see what portions of the building that they're actually utilizing and do more direct in particular marketing to them, like, hey, why didn't you come, you know, participate in Act growing, you know, with us who did laser tag but not actually showing.Maybe here's a coupon for you to come back and give that an experience the next time. And it was really a, it wasn't something I was expecting at first, but once it all started flowing together, those transactions started coming from multiple different areas. That's when we really started seeing the power and what we could do with those different types of marketing campaigns.Richard Moot: Yeah. I'm so glad that you mention that, because that's one of the things that, I mean, there's a period where it felt weird to like, sort of promote this to folks because, you know, it's like it sounds like, oh, man, this is like, you know, high level, amount of tracking, but it is like, really like, you know, I'm I'm gonna throw out some assumptions here.Please feel free to correct me if I'm off-base on how this is working. But, you know, you can see a particular credit card come in and say you have no idea who this person is when they first set foot in the door. They may or may not have a reservation for bowling, but they go up, they go to the kiosks and they're like, oh, I'm going to go get some game zone cards for my kids.And, you know, like send them on their way to go, like play in the arcade. And then I'm going to go buy some food. And you're seeing this credit card making each one of these purchases. So you're already creating a profile of this particular customer. And then when they go to say maybe they paid for the bowling online and you might already have like, you know, have this profile.As soon as they do that first transaction, you're like, okay, we know we now know this person's here. You actually know how far ahead of their appointment they're there because they're immediately starting to do these purchases and like leading up to maybe their reservation time. And it's just like one of those things where like when you have this integrated throughout your entire system, you realize, like, you can see so much more about customer behavior.And it just sounds like so much more interesting in a place where, like, there's like all of these different opportunities for like, you know, the go carts, the arcade, the food, the, the actual bowling itself that you can like really sort of see core to like the main things that certain cohorts tend to gravitate towards and like, I know that we, we, you know, like a pre call, we talked a little bit about like how you can like build different marketing campaigns of like, hey, how do we encourage a little bit more of like go cart usage.Maybe we could send you something a few hours or a day or two before somebody like reservation to be like, hey, here's a promo for doing this while you're already at the venue. So it's just really powerful.Eric Osborn: Yeah. Especially with the we do a lot of that through the smart groupings now leveraging that data. And when they were, they're left in particularly what category of stuff did they actually purchase. So you know, just to give an example, the stuff that we launched yesterday was just a basic survey to guess that were bowling and VIP because we want to know, hey, did you get your complimentary chips and salsa, did a manager check on you?And we're already seeing within a 24 hour period, that we just launched it last night, dozens of these surveys coming back of people wanting to give us the feedback, and we're just simply sending them a Square gift card in return. So we're gathering data. That's a very powerful to the business and how it operates, and where we can fill in gaps of where we could potentially make more revenue producing opportunities.And we're starting to feel it taking some time to leverage all that data. But it's working for us. And one of the things that actually encouraging them to give us more of that data is like our rewards program and thoughts. So you mentioned you're following the credit card, but most of our customers there's a good 20% or something now, which is really good for rewards program.They're electing to give us more information. So we're getting their name, their email, their phone number and their birthdate, and they're staying in a part of our text programs. But because they want to get the rewards to come back and do laser tag, or if they've never done act throwing, come and do that. So we're leveraging multiple different things that are starting to come full circle.And I don't think we realized that as a company a couple of years ago, I think you can even ask, you know, any one of our management teams, you know, what was the goal here and what did what did you envision this coming about? And I don't think they understood the full scope or the full circle, but we're there and we're now seeing the result of it, which is awesome to see.Richard Moot: Yeah. I'm curious like to, trying to like, think of how exactly to phrase this, but like, you know, it started out with something fairly simple, like, you know, the curbside pickup, you know, during during a time where, like, you know, this is the this is the way to keep the doors open, keep some people employed is doing this like curbside pickup thing.And then it kind of grew from there. I'm sort of curious about that evolution as you started to sort of plug Square into various other places, like how is that sort of mixture between stuff that you just sort of like had off the shelf stuff that you've extended on Square? Like, I guess I'm trying to like sort of understand, like, you know, how much like stitching together of things has gone off over time or is it like, really just like, oh, we just found like something within Square that we can actually replace something else that we were using and then just do like smaller extensions, like, I mean,Yeah. How does that sort of grow over time in terms of trying to get Square working with any of these other tools?Eric Osborn: It is I think it was, you know, we started with the simple stuff. Obviously, off the shelf, the food and beverage was the easiest to to get up and running because that replaced our, our, existing cards. Any kitchen printers, any more printers we had. That was a very off the shelf move for us. So right off the bat, we were able to move, you know, about 15 to 20 terminals per location over to the new system.And our staff caught on to it quickly. It was just an easy flow. And then as that evolution started coming back around, we just realized that any time that we got that data into Square, it was making all the back end stuff so much more easier. So we talked about marketing. We talked about the web financials and all that whereabouts we were piece mailing it all together.So if we needed up and from our front desk or we needed something from our our kiosks, we were writing custom dev to in order to do that, we threw out almost all of that and went to basically writing Square orders, and it simplified the whole process of not having to have our own categories, not having to have our own financial mappings.So everything we're doing as of, you know, and we might have probably just made the decision at 2025, but we're going back and looking at why isn't that in Square. So if it is currently outside of Square like can we get it in there? And if so, how can we do that? And that's where Alex comes along. And utilizing all those APIs and and the resources that are available to us, to see how we can accomplish that.Honestly, the main stumbling blocks or something, most of the time that third party partners are POS is that you have to work with because you got to make sure that they at least have the open mechanisms to be able to grab the data that you need and push it to Square. If that's not there, you know, it's totally out of the question. But, thankfully we've had really good partnerships with, with all of our plus companies and we're able to accomplish that.Richard Moot: Yeah. That's great. I mean, like, it's one of those things where like, it feels weird to, like, start having the suggestion as a software engineer to other folks when you when you're basically sort of looking at the entire landscape of your business and saying like, okay, like in order for us to answer, say this particular question, it requires me to pull data from like 3 or 4 different places and like they're all in different types of formats.And then I'm having to sort of do some data munging to like, kind of like, well, these are done in like the wrong time period. So I have to adjust it in order to work in the same time period as this other data set. And like once you actually homogenize everything into like a singular system, you're, you're writing less like throwaway code is like kind of like what I think of it because like, these one time reports are like maybe quarterly or biannual that you're just like, oh, well, we really need to answer this particular question because it's going to let us know, like, can we run this marketing campaign and expect it to be successful?Eric Osborn: Yeah, I mean, there's dozens of one off code that we've been throwing out over the past couple of weeks just by pushing the data into Square. You know, I keep, you know, running into more and more of things that we're moving over. I said, we just moved to surveys and stuff over. We just moved to another POV.But not even a week ago, while I was IPO, Alex moved to a game zone system over to Square as well. So yeah, we mentioned those games on cards that you're buying at the front on the kiosks. We're actually now grabbing the data from the video games, and every 15 minutes uploading those transactions to Square as well.So we know how much video game play versus virtual reality versus laser tag. All that's now being funneled in. And that was a, I think for Alec that was, maybe a two day project. And we have another POS system that was linked in the Square. It's just it's been that easy for us.Richard Moot: One thing I do want to kind of circle back to just, because I'm, I'm interested on a, on a technical level with the kiosk that you're using. If I recall, this is, like a kiosk system that you got through a vendor, but, the device itself, it's running on, like a windows OS as its base, which is kind of like part of the motivation for using terminal API versus like, say, our mobile payments SDK.Yeah, yeah. Tell us a little bit about building the integration with terminal API and this device. Because from what I've understood, like, you know, thankfully the device that you have is like, you know, fairly open in terms of being able to plug and play certain things with it. But I'd love to just, you know, tell us a little bit about, like, the app that you built and how that sort of works and communicate with this kiosk.Alex Trepasso: Yeah. So I'll, I'll kind of start from where when we first started designing it, where I jumped off from was, I know we had provided by the kiosk different payment methods that I could do. There's, you know, ones that are hosted on site. There were cloud providers. They had, none of which were Square at that time.So we found a provider that they had set up that actually was using a Soap protocol to send out its transactions to what it assumed was a local server. And that transaction information was just a little here's the bill ID, here's the amount to charge, you know, charge it and move on. What we ventured from there was kind of, almost pentesting that of, okay, can I intercept this and act as that receiving server?From there, it jumped to, okay, so now I got the request. I can take it in, I can serialize that. I know what my amount is. I know my bill information. I know what response it's wanting or expecting. That little piece in the middle that we've named, Mercury. Just because I used record naming for a lot of our software just for the fun of it, basically acts as just a messenger in between.So it takes that so request translates what the charge should be, what any of the bill information sends that to the terminal API to create a transaction. And then basically sits and waits for the terminal to reply or that transaction update and say, hey, I'm complete or I'm canceled. From there, I translate that back into what the kiosk is expecting as a payment response that just says, hey, was it a denied?Was it rejected? Approved? What's the status? I tell the kiosk, yep, we're good. We took the payment and it moves on just like it would request a normal vendor or the normal system that it thinks it's talking to. Basically just that messenger in between. So there's no creating of orders or trying to mess with the bill in any way.It's just going, hey, here's my amount. Translate this into something that terminal API can understand, and then translate the terminal API response back to something that the kiosk can understand, and that that became the philosophy for a lot of the software was instead of trying to reinvent the wheel or make these throwaway reports or that throwaway code you were mentioning, we go, okay, what if we turn these into basically data pipelines and let Square continue to do what it's really good at with the reporting and the transaction handling and just get the data to Square, let Square do the actual work.We just act as kind of this in-between data integrator of saying, see, here's the data Square can do whatever it needs or whatever it wants to do with those numbers for marketing, loyalty rewards, business tracking, anything like.Eric Osborn: That, unless you're currently working on it. Good to mention that this integration was done well over a year ago, before we thought that, you know, Square was going to be that total ecosystem and where we were going to funnel all of our data. So it basically started as a custom amount integration on those kiosks. And Alex can speak a little bit about what we're doing now.He's working on the full itemized integration for that kiosk as well. Now that we see the benefits of what we need to do there.Alex Trepasso: Yeah. With the, I'd almost compare it to a snowball if we started very small in control and we started with this one off kiosk integration. And then as we integrated more systems and more features of the different systems, I realized and through discussion with Erik, realized we're doing a lot of the same actions over and over again.So instead of designing, you know, five apps that communicate between the five different pieces, what if we do one that understands data and knows how to get it to Square? And instead of building everything from the ground up, every time we just build these essential sort of intakes of data. So we take the data from the POS, let this centralized system convert it to what Square wants and what Square's looking for, and open that door to when we have a new POS.We're not trying to rebuild everything, we're just doing something that integrates. We got the data in this format. We already know Square wants this. Let's just take it and format it how we need, which is really spelled out development times. It's allowed us to, like Eric said, turn around an entire new POS onto the system in two days, rather than, you know, having to go all the way from floor zero or ground zero of where am I going to go with this?Now we kind of have a set template of every time when we're moving forward. Yeah.Richard Moot: Now that's very cool. I mean, like, you know, I had this, like, silly. I thought I might have this, like, this, like this Borg, like a simulator. That's just as you bring it in, like this new thing. It's just like going to go, I'm going to assimilate you to, like, funnel the data into Square and make this unified.Eric Osborn: Yeah. And we would say no at this point if it couldn't funnel in the Square that but the amazing part about it, and the only reason those systems exist are because they're specialized systems, like a game zone or running a go kart or running bowling. So those are always going to exist in the entertainment world.Richard Moot: I mean, like that part I still is, just like I feel is like very fascinating because, like, you know, I probably one of the few people in this world that, like, will ask random people who own a business, like, you know, what is it that you use to, like, operate various different components and like in an entertainment center like this?I'm just surprised, like how many different types of systems that you need for managing different entertainment parts. Like, you know, I mean, most of the time we might just, like, take it for granted, like, oh, yeah, you know, they got cards for the arcade, you know? But there's got to be a whole system for, like, you get the card, you got to load the card, you got to track how much is on the card.It has to communicate with each one of the game machines, which is then also going to communicate back to a server to say like, hey, does this person have credits or multi-location? Yeah, yeah. One thing I do want to touch on is even in like, you're, you're handling payments for the game. So cards you originally were had this approach where you were sort of like sending the revenue directly to your accounting software.But then part of this, like migration you had like this, you know, win of going by having this go directly into Square and Square, then now sinks into your accounting integration. Yeah. And so that's kind of like sort of saving you even from having to deal with sending your stuff directly into the accounting software.Eric Osborn: Yeah. We actually had some of those one off development things that were going on because a lot of them didn't have that native integration to the accounting platform that we were using. So Alex had these, these one off tools that would run in the middle of the night, basically exporting, you know, text or CSV files that could be ingested into the accounting software.So we've eliminated all those little one off things. And as Alex was saying, it's kind of all using the same mechanism now for ingesting the data, throwing it into a Square transaction. No matter where it's going. So just overnight we, we eliminated all the actually within the past couple weeks, 3 to 6 different little one off, import export tools that are now just being ingested into, Square.Richard Moot: And I mean, it's, it's like, I feel like you're just, like, telling my, my dream story to be telling about the ecosystem because, you know, we we love to talk about the, the ecosystem that we provide here. I think sometimes it's even hard to, like, really understand, like what the ecosystem of products really is. And I think you have like, just like the right mindset to, to be able to see like, oh, like, you know, it's working in this particular area and like, you know, it kind of could work for this other thing.Let's try it over there. And like you know, you solve the small problem first and then you kind of realize like, okay, well, now that we solve the small problem, we realize we can kind of have it solve more than just that. And you keep growing it and all these different areas and eventually it's just like you're now at this point where, like, if it doesn't work with Square, we're not really that interested.Eric Osborn: We're not going to use that. Yeah. Yeah, it's kind of amazing now that, y'all, I find myself scrolling through the dashboard or transactions. What funneling through there is pretty incredible. Now that we can link all these transactions together. And our staff and our managers, our team, they all know that, you know, anything that they need to manage now, as far as refunding or voiding or crediting a customer, they go right to a Square tablet.It doesn't matter where they bought it in the building, whether it's a kiosk, a game or online or bowling, it doesn't matter. They pick up a Square tablet and you now have every single one of those transactions. Right there would be for us to go into three, 4 or 5 different pieces of software just to figure out where to refund it.And at one point we were going into multiple credit card processors to figure out, you know, what went wrong or what needed to be refunded. So I find myself scrolling through the dashboard every once a while ago, and while this was actually really starting to come together for us.Richard Moot: So I think the final thing I do want to touch on because, just to geek out on, is you built, the integration for the kiosk system. So you're running that, I'm assuming somewhere locally, or is that actually like a cloud solution? It's like connects with the kiosk software that you're sort of having, like, I guess I'm more curious.Like, what did you build it with? You have like a little server that's probably like sitting there listening, waiting for, you know, something to come through. Tell me a little about your tech stack is really what I'm getting.Alex Trepasso: Yeah. So the first point I kind of went with was knowing that it is a payment provider. It needs to be incredibly reliable because you don't want a customer having a failed transaction, as typically, at least with me, within FCX, a failed transaction or a pain point will push a customer to no longer use it. It's kind of an all or nothing of if it's difficult, they just simply won't go through that flow anymore, regardless of what the end result would be for them.We then, using our cloud provider, found out we can integrate it directly into our network essentially. So while it's a cloud hosted server and they handle, you know, the uptime and making sure it's online, making sure it's reachable, making sure it's, you know, healthy as an app, it's integrated directly into our network. So that kiosk is sending out a request to what it thinks is a local server running in the building, that goes through our networking system and then gets to our cloud provider.It processes that request and sends it back, without really any points in the middle that neither system thinks it's in the cloud. Essentially, they both think they're working locally and they both have direct access to these resources without any kind of pain points there.Eric Osborn: It's funny that Richard mentions the, you know, the computer sitting in the office or the computer, the server sitting in the server room, because that is where a lot of this stuff started. When I first started and where Alec first started, there were a lot of these one off little servers, you know, sitting around. But, like he's saying, it's going through an SD-Wan solution at this point to the cloud provider.So it's just there and there's multiple redundancies in place for that, to make sure that continues to happen in any way, shape or form, you know, for losing a number, we might switch over to broadband and then broadband. Unfortunately, we don't have great 5G connections, but we could switch to a 5G connection if we absolutely need it to.Alex Trepasso: And I think, on the tech stack, a little bit of that app that provides some of that is we knew going in, we didn't have, you know, time to reinvent the wheel. We needed something that we could kind of develop quickly on, iterate quickly on, you know, if something needs change, change it, you know, on those one off scenarios.So, that was kind of our first dive into Node.js and using TypeScript, Node.js and essentially an express server that's processing those requests and allowing us to kind of quickly get things working. And I got to give some of that credit to you guys over at Square for that your API is very developer friendly. Probably one of the most developer friendly we've worked with in our different vendors and with its multiple different kinds of integrations, both with its direct SDK for Node.js and its direct different integrations with these different platforms and tech stacks.We're able to, you know, take something that normally would take a week because we're designing everything for it and quickly iterate and go, you know, overnight turnaround and have a solution to integrate these different technologies. And that's been carried forward into pretty much every software of the Square API has been very developer friendly and very open to working with different data formats that we've been able to quickly spit out these integrations and get things up and running without, you know, huge turnaround times.Eric Osborn: I mean, that chaos integration. And looking back at almost all the integrations, it was a 1 or 2 days of dev, and we're writing the testing and, the chaos integration was one of the most flawless ones. The testing from day one, there wasn't one failed transaction. And then going into a weekend that has, you know, a thousand transactions in a week and then not having one failed was pretty amazing.And that gave me the confidence and the entire team, the confidence to continue moving forward with this.Richard Moot: I love to hear it. I mean, like, it's it's one of the things that I, I've heard time and time again. I feel silly because it feels like I'm just singing our own praises here. But, the ability to just, like, sign up for a developer account, get credentials, and immediately start building and testing. I can't tell you how many times, like, somebody, you know, started out with an idea where, like, they were at, like a baseball game or like the coffee shop and they see the payment system there and there's like, wait, what is this?Can I and then they go look up the docs and within a week they have a proof of concept up and running and they're like, hey, I already like having a terminal device taking a payment. And, you know, they're shopping around within their company to be like, hey, we can use this. Like, this can actually solve the problem.And of course, like, you know, eventually have to build out to scale. So, you know, you go through the process of getting Docker-ized apps in your cloud provider and getting, like everything strung up. But like, you know, you have that moment initially where you're just like, no, like, this can work. Like I've, I've always believed in that.Like for developers, like you can convince them, like in that moment, this can work. It's working for you right now. Like, yeah, you got to build out the scale and the reliability. But it's going to solve the problem that you want.Eric Osborn: Yeah, it's even more exciting when it's, when there's a known pain point in our operation that we can bridge. And a lot of times, you know, the staff, customers or management won't even realize the bridge is there. Making that a simple process. Where about if you go to, you know, a lot of the, the one off folks that are single train operators and that they might not have, you know, had somebody to be able to take that leap for them.So they are still doing it all that multi transaction and the multi POS, it's really a seamless process at this point.Richard Moot: Yeah I mean you know going going back to like what you touched on earlier of that still surprises me to this day of like you know a lot of businesses in or like other software providers or hardware providers for that matter, you know, have this expectation that you might have this like literal server setting in like a back office somewhere that's going to be connected to your network.That's like, no, it's I'm going to try and find this thing. I mean, like, that's the whole reason that you're able to build this, this integration with the kiosk solution because, yeah, a lot of businesses have that. I've talked to that. Clinics, law firms, some of them have to for regulatory reasons like, you know, their system data has to be in this very controlled format.But it makes these integrations like a lot more difficult when, you know, you have to then go talk to a vendor, get special documentation and like try to figure out like, how does this system actually work and how can we make it like work versus like, hey, here's like an open system where you can just immediately go in and build and test.Eric Osborn: And that way a lot of our decisions at this point is looking for that open API. And what can we leverage and what can we do with it, rather than focusing on the front end, you know, the gooey and what that's actually happening is if we do need to do the one offs, or we do need to bring it over to Square, is that available for us to do so? And that's important at this point.Richard Moot: I mean, it makes a lot of sense, you know, for something like, you know, an FCC like, you know, it's a very you want to have it how you build your business. Because what you build is very unique. You know, it's you know, it's your particular mixture of like having the go carts, having the arcade, having the bowling.Like, you know, you're going to be able to, you know, find a solution and build something that works for you guys instead of just trying to like, you know, take stuff that's off the shelf and, you know, hobble it together.Eric Osborn: Yeah. And our team hasn't seen it yet. But, you know, we're trying to focus a lot on selling packages and bringing the different attractions together so that when you're just coming in for laser tag, you're just coming in for bowling. It might be a little bit more enticing if you're, you know, getting 50% off on your next attraction by bundling it together or, you know, as I mentioned before, we'll fast Tracks will be our go kart place across the street or across the parking lot, essentially.So you're going to walk in the Headpinz, you're going to be buying bowling, but you're going to be able to add racing to that all through one ecosystem. So you're going to be in one location while booking at location B, and that's all made possible by pulling these pieces together.Richard Moot: Yeah. I mean it's like software. Is it like literally enabling the cross-sell?Eric Osborn: Yep. Exactly.Alex Trepasso: To jump in on that. I think a big part of that, at least for us, was a lot of these other vendors, you know, there's vendors like that do our games that have been in this business for 25 plus years. They know what they're doing. They've mastered their little niche. And while it may not be something that Square provides or would go down the path of writing, we can make that work because Square is very open to understanding.Yeah, there are going to be things that, you know, especially in an FCC type environment or these large business environments. You have these multi different essential ways that a product gets old. You know, you have bowling. That's my lane. You have go carts. That would be by the time slot. The arcade is just money on a card that are all very different.And how the user experience is in the business view is that just wouldn't exist together, that we can make work together. So for a customer, you know, they're checking out in one place and having that singular transaction rather than having to go through five different transactions to achieve the same result.Eric Osborn: And it makes me think on the In the Square catalog side, like, one of our providers for bowling, they simply just added a custom attribute that decides is that price key for bowling shoes, price fun, whatever that may be. League bowling with just a simple dropdown that decides where it goes into that front desk. For bowling wise.So it is really that simple when it comes down to adding those keys and utilizing them.Richard Moot: Yeah, I really want to reiterate like what you said there that is very important to like sort of highlight for other folks or just like the different ways that you sell things. I have come through this time and time again, like just dealing with a catalog is, you know, different types of businesses have different products that they're selling.But like, you know, bowling is by the hour, but like the shoes are like Purge Day and then like, you know, you're going to have all of these various, like, different things that like a very bespoke way of selling this particular product. And so having a flexible system that allows you to like, unify that and can meet your needs as you have different things you want to add or offer, and there's a programmatic way to do it.I do want to thank you both, Eric and Alex for joining me here in the podcast. It's been very enlightening hearing how you have such a tech forward family entertainment center business. I would recommend anybody if you're in the Florida, Southwest Florida region. Definitely go check out Headpinz. You can find us at Square Dev on X. Or you can reach out to us on our discord. And if you have a project that you're working on, make sure to let us know about it. And don't forget to subscribe and check out developer Dot Square up.com for more. Keep building and catch you next time.

  --------  

  43:03
• Integrating Phone Numbers into Brand Identity Verification

  Richard Moot: Hello and welcome to the Square Developer Podcast. I'm your host, Richard Moot, head of developer relations here at Square, and today I'm joined by Binh Ly who's a member of our developer community and is the owner and operator of the company operating. Ben, thank you so much for joining us here. I'm so excited to chat more with you about what it is that you've built on the Square Developer platform. You're also a hackathon winner. I'd love for you to just tell us all a little bit more about how you first got involved with Square and a little bit about your involvement on the Hackathon.Binh Ly: First, thank you for having me. It's a pleasure to be here. So Square has been on my radar since around the time of the company's founding. I was working with a device that could handle credit card swipes, but I wasn't using that component of the device, but I was trying to think of a reason to use it. And then back then the thought was like it stinks is when you go to the restaurant and you're with your friends, so why can't you split that check? So we're like, let's try and build something to do that. But back then onboarding a merchant account was not that fun. So that lag time and making the sale and then getting the software activated for someone was too long, but we were fascinated that Square could do it in two minutes. So I was like, Square is pretty interesting. So I just followed the company's trajectory that whole time. And then I finally switched careers since I changed the thing that I was working on from shipping software to messaging software around 2017. So that was the first version of Operator that existed in a different company. And back then the idea was that you should be able to message any company, but how do you do that without selling software at every business in the country? So we had this really insane approach where if you text it into the system, we would call the business and ask them the information and then text it back, but,Richard Moot: Oh wow.Binh Ly: That was pretty neat that it worked that way, but ultimately wasn't scalable.But then once we realized that you could send text messages to the landlines, that changed everything because a majority of businesses have landlines and SMS is the most installed software on earth. So to get the customer you didn't have the two sided problem, the software was already on the phone. We just need to collect the text messages sent to the landline and present it to the business owner.Richard Moot: I mean, I always think that that part is fascinating to me because it's still, even after you submitted the hackathon thing and every time I come back to it, I think this is something that most people just don't think is possible of getting SMS on a landline. So for those who are not familiar with this, how is this able to work or to what degree could you sort explain how this works to us?Binh Ly: Yes. So the way it was explained to me about how it works is that you can picture a gigantic phone book and there's every phone number, every landline number is in there, and there's imagine two fields next to every phone. Number one says data traffic and one says voice traffic. So when you get a cell phone, the voice traffic says whoever your carrier is, AT&T, Verizon or whatever. And then the same for the data field, but for a landline, the data field is empty. So when you send a text message from your cell phone to a landline, it just goes to nowhere because it doesn't know where to route it. So by taking over provisioning the landline for voice traffic or data traffic, we're saying route that to the operator system.Richard Moot: I see. And so does this require any kind of physical component or is this actually something that's like they just, it's done within at the carrier level?Binh Ly: It's all at the carrier level.Richard Moot: I see, okay.Binh Ly: Yeah, so to actually utilize this capability, you just need authorization from the business that owns the landline. And so they just signed their name on a letter of authorization and we submit that to the carriers and then a few hours later, traffic starts flowing through. So that's one of the moments of delight for the customer is that they didn't realize this was happening. They signed up and they started getting traffic coming in and they're like, I didn't tell anyone we can do this. So I'm like, it's already happening all day every day and now you're getting access to it.Richard Moot: So what you're saying there is have some businesses signed up for this and suddenly without even prompting of people are getting text messages and didn't realize that people were texting them this whole time? Yes. Wow.Binh Ly: YesRichard Moot: WowBinh Ly: So a lot of missed business happens this way. We just saw, we signed up a window tinting business without telling any of their customers. They're getting requests, can I get a quote on this tint? If they didn't have the service, they would not know about that requirement. And the other cool thing is that it's not replacing anything. You get the voice traffic still and now you get the data traffic. It's up to the business owner how they want to respond. They can call 'em back if they want that hyper personal touch or just text back, which majority of the customers who interact with the platform prefer that just text, send text, and off you go.Richard Moot: I mean, I'm guilty of that. I've been so stoked in the recent years that more and more businesses are allowing texting directly to these lines because it's been able to reschedule. I mean, giving my car serviced at this local car shop, it's so much easier that I get a little text message, Hey, your car's ready and you text back and great, I'll be there by two and I don't have to answer a phone call meetings all day. Most of the time I just let that go to voicemail and then I hope that I can text 'em back to say, Hey, I'll be there. So it's so convenient.Binh Ly: That voicemail aspect is also a really interesting component of the customer communication. We found that people don't leave voicemails. So if someone calls you miss the call and then they don't leave a voicemail, it's really hard to have any context around that. So we invented this technique where if you call the landline, someone calls your landline and if no one picks up, the system will text you back from that landline number. So it's not a spooky experience with a caller. If you called one number, then you got a text back from another number saying, Hey, I saw that you called. It's kind of a weird experience. So you probably ignore that too. But if you call one number and you get text back from the same number, the response is pretty immediate.Richard Moot: Yeah, I mean it's so validating because like the phone number for a lot of businesses, this is a form of identity for them. And so by keeping it all within that one thing, it feels very cohesive. You can trust it. I've definitely had this spooky thing happen where I call one place and then I get a text back from another and I'm like, I feel like I'm being, this is a weird phishing thing going on.Binh Ly: Yes.Richard Moot: You definitely don't want to have more of that.Binh Ly: Exactly. And a lot of businesses don't realize until you point it out to them that their phone number is part of their branding. So if you drive by auto shops or something, you'll see the name of the business and sometimes the phone number is just as large in the typeface. So now you can maximize the utilization of that number, put a simple call or text this number in. In comes the traffic.Richard Moot: And so you have this technology of being able to do this SMS routing, and then you built an app for R Square hackathon and you won. Tell us a little bit about more what you built and why you built it for the hackathon.Binh Ly: Yes. So the project that won the hackathon was not the project that I started with. The initial project was digital appointment cards. So if you book through Square Appointments today, you'll get an email and in that email are a couple of links to add to your calendar. That's over email. But if you think about in real world scenarios and medical practices, for example, or dental, you go in, you do your checkup or whatever at a dental office and they say see the front desk before you leave. So you go out there and they say they want to see you in six months, they put it into their system and then they write down your information on a card and hand it to you. So it's on you to either remember that forever until the appointment or enter it into your own calendar. So I didn't know at the time that Square sent that email that lets you add to the calendar. So I thought, let's do that over SMS, but I built that those tied in Square Appointments and it sent you a text where you can add it to your calendar. Then we saw that you could do that over email. I was like, oh no, this is the saddest thing, all this work. Then I vaguely recalled an article I saw on your website about the number of no-show and cancellations,And it reminded me that it's like the missed call thing. Again, if you don't figure out how to respond to that, you're going to lose it hundred percent. So we looked into how do we, could we hijack that process? If there's a no-show or a cancellation, can we help the business try and recover that? At least take a shot at it, right? 50/50 chance of getting that business. So that was the genesis of that.Richard Moot: And to be honest, we were very inspired by that because being able to recoup even half or a quarter of those no-shows is that could be the difference between making or breaking it in a given month or quarter for a business. And I think the other part that you had mentioned previously that was also really impressive is say somebody has an appointment, they called in, they didn't leave a voicemail, maybe they wanted to reschedule, with Operator, you can actually just send a message back like, Hey, what did you need help with? Or maybe even be smart and look up like, oh, does this person have a booking? Were they trying to reschedule? There's so many different kind of workflows that can spring from that touch point.Binh Ly: Yes, and I think the recent advent of LLMs has really helped and how that is experienced because the hackathon project, and this goes back to why people pick Operator over other messaging providers I guess you'd say, is that in the hackathon you can project, you can see that the cancellation campaign, when it sends out the sequence of text messages, you can customize all of those to use your own language. And the interesting thing is whether we've been experimenting with using LLMs for is it turns out it's kind of hard and people also manage to mess up reply Y to confirm, or one to confirm they can fat finger that easily. We see it a lot in the logs when we're helping the customers diagnose why some of the automations aren't happening, but you can feed that into the LLM, like, look at this conversation, do you think they confirmed or not? And it turns out it's really good at that. That's one my favorite uses of it so far.Richard Moot: And so the other thing I wanted to sort of touch on is that you have this product follow-ups, a part of Operator, but this is actually kind of grown beyond just sending messages for missed appointments, but there's many other use cases. I was hoping that maybe you tell us some of the other use cases that people have had for this communication style.Binh Ly:Yeah, so like I mentioned earlier, the customization of the messages is one of the big reasons people keep contacting us, and that's not just over SMS. So one of the complaints, not complaints, I guess the feedback that we get about built-in appointment reminders is that they're all the same.So we allow our customers to customize that initial message and the follow-up message so it feels organic and on brand with how they do business. But that didn't exist at the time of the hackathon project. And the other thing is that this capability has found, when we first envisioned it, it was for stylists, people we were most aware of with who deal with cancellation and no-shows, but the general awareness of Square's, APIs, the Square appointment products, Square payments has really opened the door for Operator to kind of transition into a consulting business. Someone will say, I have this technical need or this problem, and I can say, okay, I know how to solve that with some software and very specific integrations.Binh Ly: So Operator is being white labeled essentially next year in an unexpected way with a company called BeamReaders, and they're a dental radiology company that will be using Operator’s infrastructure to communicate with patients. They have this new, this need where currently they operate kind of like an Uber for dental radiology, someone who needs a ride and Uber matches you with a driver. So in the current business for Beam readers, someone has a cone beam scan of their head, they need a radiologist to interpret it. So BeamMeters connects the two.Richard Moot: Interesting.Binh Ly: So now there are a plethora of these cone beam devices across the country, underutilized. So they'll be launching kind of like an Airbnb, but for the cone beam.Richard Moot: So for anybody who's not familiar, give us the quickest way to understand what these cone BeamReaders are, what they're used for.Binh Ly: So typically you go to the dentist and you get an X-ray done. That X-ray is in 2D and dentists go to school and they're trained to interpret 2D, but the cone beam technology is 3D and it captures more of you, not just a chunk, so maybe from the chest to the top of your head. So in that range of coverage, there could be diseases that the dentist is not trained to identify or address, but by law they are liable for that. So if you go get a cone beam scan, there's some cancer somewhere in you, but the dentist doesn't tell you they're liable for that.Richard Moot: Wild.Binh Ly: BeamReaders exist to shift that liability, so you send the raw cone beam files to BeamReaders, they have radiologists on staff who will read through it and then write up a report, let you know that there's something wrong or if it's all good. So that's the core business around that cone beam technology and if they're getting adopted in other industries too. So we're seeing it happen with chiropractors. So if you have a 3D view of someone's neck, that's really powerful and helpful.Richard Moot: No, I mean as somebody, I have a pinched nerve in my neck and I had to go in, get an MR, I get steroid shots and I'm sure that somebody, at least with a better analysis of doing a 3D of my neck would see like, oh, is it the actual bones? Do I have bone spurs? They could rule that out much more quickly, and so yeah, I mean I definitely see there's plenty of use cases thereBinh Ly: And that technology is being installed all over the place, but it's very expensive. So by allowing other businesses that don't have a cone beam to kind of rent the cone beam for a bit will just help grow the adoption and care coverage, essentially. More people will have access to this because of this software, but they're thinking about using Square Appointments.So if a practice has a cone beam and they want to sign up for BeamReaders Airbnb type product, they're thinking maybe they can also sign up for a Square Appointments account. And those appointments, if someone's referring a patient to your practice, you can put that into the Square because it's not really your, so it doesn't make sense to put them into your patient management system. So you put them in maybe something like Square and then facilitate all the communication over SMS. Because in my experience with Airbnb, at least, you book the place, you send the initial message to the property owner, but when you check in, they're switching the SMS right away or iMessage, no one wants to log back in.So the same thing we think will happen with the dental stuff. The patient does not know the practice that they're going to get this cone scan. The more friction you can avoid, the better. So just send a text message directly to the patient from your landline and you'll cover that communication gap immediately.Richard Moot: Yeah, so one thing I want to circle back to is that issues around the liability, and then I'm curious on the Airbnb type component to this. Is that in part to maximize utilization of the cone beam? Because from what I would understand, they're probably not cheap. So you still want to recoup the cost of it and maximize the use of it in order to make it worthwhile to have. So is this to make sure that people can find out where these things are and get these scans done? And then also to touch on the shifting of the liability there?Binh Ly: So there's actually two parts to the answer to that. So yes, the cone beam device is very expensive and dentists will acquire a machine and they don't always use it because of the liability risk they have to pay for the report. So to provide the most amount of care, the best care, if you have this equipment, you have to incur a cost as the provider.So the Airbnb model will help, hopefully at scale help to reduce lower that cost because you're recouping some of that. But the other thing that BeamReaders is doing that will be doing that's pretty innovative is that they will allow the provider to offer the patient the option of paying directly for it.Richard Moot: I see.Binh Ly: You'll go in, they'll say, look, we took a cone beam scan of you, but to ensure that there's nothing wrong, we recommend that you get it reviewed by a radiologist. If you want to do that, then they'll slide a Square terminal in front of you, the patient sticks their credit card in, and then the appropriate charges and fees get distributed. Everyone wins in that scenario.Richard Moot: So shifting the patient in this instance is paying for this directly, that kind of shifts the liability that's like no, it's their scan that they're paying for rather than something that the dentist does and orders and runs through insurance.Binh Ly: Yes.Richard Moot: That kind of helps in sort of shifting the liability that it's now on the patient to sort of follow up with a radiologist or others?Binh Ly: YesRichard Moot: I see. Well, I'd love to see the Square terminal device, just put it right in there. I know we have plenty of workflows to make that a little bit more seamless, but yeah, that's really interesting that you can just do one simple change in the ordering of things or who's paying for it can make that far more viable. Because one thing I know that we had talked about before doing the podcast here is the actual impact that this particular technology can have for people because of what it can see that you normally couldn't. I don't know if you want to sort of speak to that a little bit more.Binh Ly: Yes. I asked, I think recently, what percentage of scans come on a day-to-day basis. When a scan comes in, how often do you see something dangerous? And the staff said every day, multiple times a day, people are just getting their scans sent in and there's cancer or something else terrible. But it's really wild to think that only a small percentage of the scans get sent in for this kind of review. That's really scary. So anything that we can do to make it easier on all the parties involved to be like, this is a good thing to do, you should do it all four. And it has really huge implications for child development too. We've seen a couple of examples where you'll see a child, it's like bags under their eyes, they're lethargic, bad, poor grades, they go get a scan, you can see the airways obstructed. So their devices that you can give to the child to help aid in the development of their face improve the airway. And after these things are administered six weeks to three months later, no very noticeable change in the child's appearance and behaviorRichard Moot: And all just from being able to have a more detailed view of the whole passageway from using this 3D scan.Binh Ly: Yes, and I used to really, I've been scanned twice and what I was able to see about myself was really eye-opening. So Asian people I guess naturally have smaller airways, but my airways are exceptionally narrow. It's 72 millimeters.Richard Moot: And what would be sort of a normal to just give us an understanding of the difference between 72 millimeters.Binh Ly: I doubled that.Richard Moot: Okay. Wow. Wow.Binh Ly: Yeah, so I remember one year I put on my Christmas list to be able to run a mile. So I thought it was all kinds of fitness things, but it really is obstructed up here. The oxygen can't come in to begin with.Richard Moot: Yeah. Wow. That's something I just wouldn't even really think about.Binh Ly: And it's only possible because we have good tools now to back up someone's educated guesses.Richard Moot: Yeah, and it has me wanting to go get one because I've been diagnosed with a mild sleep apnea and I'm sure that it at least helped me rule out, do I have any kind of actual obstruction in here? Is there something that can be done about that? I'm sure something like that would be super helpful for that.Binh Ly: Tremendously helpful. It adds up the longer you are apneic, the more damage to your body.Richard Moot: Yeah, no, I mean I've learned over time, it feels silly to say this because of course sleep is important, but it's when you don't have good sleep that you start to learn how really, really important because it kind of just affects every other component of your life. And that's why you talk about these children who are having obstructed airways and it's just causing a perpetual issue and they're at the most critical stage of development that to have an obstruction is just so much more dire.Binh Ly: And as a parent, I've reflected upon the times where we thought that there was just bad behavior, but it turns out the body is signaling. So for example, bedwetting, night terrors, those are all signs that the child's body is giving to you that it's not getting enough air. So it's trying to wake up the child to get more air. I've heard from a lot of parents, yeah, they just can't stop wetting the bed or they're frustrated for some reason their child is not developed, like all the other kids, I dunno, why they can't grow up. It has nothing to do with that. You got to look at the airway.Richard Moot: So weird segue, but I do want to circle back to some of the stuff that we talked about earlier with, because you mentioned about LLMs and I do recall we had sort of touched on some of the ways that you sort of evolved your product over time because as we know that SMS has become more regulated and so being able to go and get an account, I mean I think some of us has had this with the various API messaging companies, you have to go through a little bit more of a process. But then also you've kind of innovated on ways to help mitigate certain types of issues to enable your users to use your product in the way that they expect it to. So I don't know if you want to, first let's talk about the regulation piece. How has that sort of changed things for your business and how have you sort of reacted to it?Binh Ly: Yeah, so around the start of the pandemic there was the formation of this entity called the Campaign Registry. And if you wanted to send SMS traffic from your software, you would have to register the phone number with this organization and then also specify the message payload. So give them a sample of what it is that you're sending and then they approve it and once they approve it, you can start sending messages. But if you don't do that, you can still send, but you'll be charged a higher fee. I believe, at some point in time next year they'll finally block it. If it's unregistered SMS traffic, it'll get blocked completely.And it really seems like what they consider spam traffic feels more restrictive now. And so despite that, real businesses have real needs that they need the software for. So we started to notice that for one customer that uses it as an alert system, the first person on the pool of numbers gets alerted, and everyone else gets blocked. So that was causing a lot of frustration since these are moments of importance for business and they can't afford to have these messages not be delivered. So that was the first time we ever used an LLM for anything. It was to rewrite the message, so it says given this recipient, this is their full name, rewrite this message and then send it to them. So the business, they compose the alert message however they want and then the LLM will rewrite that for each recipient and we cycle through three different providers for that. So to try to maximize, because even if you change the temperature sometimes you'll still get back the exact same message and then it'll get blocked. So it goes 1, 2, 3 anthropic, Open AI, Llama, and then just cycle through the numbers that way.Richard Moot: I see. And so in your case of the alerting thing, it might be like say I'm going to give a contract example because I'm not going to mention what the specific case was, but say that we had an alert that a service went down, you want to message all the engineers who were on call previously, you'd be like, it's just going to get to the first one of the list list that one goes through the rest from our blog. This would actually try to rewrite the message to make it more bespoke to each person on the list so that these look like a series of individual SMS, not one massive one.Binh Ly: Correct.Richard Moot: Yeah. I mean it's a very interesting way of circumventing that because it's like obviously there's a need here, but at the same time, I know that in the last year I think that we've all gotten lots of SMS from political campaigns or from other organizations. And so it's interesting to see how you can help a business be able to use this for a specific use case and not for something malicious, like sending a weird spam text to a bunch of random people.Binh Ly: Yes, exactly. And the other interesting use case along the same lines was that we had one customer who would send out really long messages like up to 1200 characters and you can't send that large of a message over SMS reliably because the carrier will break that up and then they'll come in out of order. So they'll compose it. Then we ask the LLM to break it down into bullet points and it can construct more sensible breaks in the long message that the carrier can. So that has both increased their throughput and if the message makes sense or not. So that was pretty neat that it can do that.Richard Moot: And so one of the other things I want to just touch on just for the sake of having people understand how to go about building basically to help inspire those in the Square developer community, what did you first build Operator on in terms of backend languages, front end stuff, and how has that sort of evolved over time?Binh Ly: Oh yes. So the very first version ever of the system was written with an elixir.Richard Moot: Oh wow.Binh Ly: Using Phoenix because it had really strong web socket support. That was 2018 I think, and we had a mobile app that also connected. Since the Apple push notifications weren't guaranteed, we also applied web socket usage there, but then the guy Elixir and Erlang and the Beam, they're kind of exotic technologies to me even to this day. So when I had to need it to rewrite the whole system, I did that in Ruby with Rails the fastest way to get it up. So it's a webhook heavy system. So that's how the carriers communicate with us. So thousands of webhook calls an hour.And so despite that, the system is still really small. It's a pretty small instance, at Amazon, we store every webhook message that comes in and process that on a background queue because the luxury of SMS is that the receiver doesn't know when the message is coming. So it's not like WhatsApp where you give them the little dots that someone's typing that's kind of unnecessary for business messaging.They just know that the message is coming, so that buys us a little bit of time. So it's a Rails app that uses Sidekick behind the scenes. So small Rails app with the Sidekick machine is kind of large.Richard Moot: Awesome. And are there any other components? So it's just staying with a Rails app. Are you using anything for any kind of front end components of this?Binh Ly: In the newest or the latest version of Operator, it's Rails. We just upgraded to Rails 8. It uses a lot of tailwind, but it tries to be as vanilla rails as possible though, because should something go wrong, I want to be able to just picture in my head where the problem could be.Because my thing is I played a lot of hockey growing up and there's this saying in hockey, if you're a defenseman and you play a good game, no one says anything. No praise or anything, you play a bad game, everyone brings you up. So the messaging service is kind of the same way on a day-to-day basis. It's quiet, there's just message traffic, no support tickets. But when there's a delay or the customer knows that someone's texting them but they're not receiving it, then it gets crazy real fast. So the system's designed too, or I implement it in a way where I can quickly diagnose or try to quickly diagnose where the problem could be. So storing the webhooks in the database for 30 days, I can always replay things.Richard Moot: Right. Well, I think you have a really, I can't speak even more to how much your approach to that aligns in terms of that. As long as you're doing everything that you're supposed to be doing, everything should be quiet. But as soon as things go wrong, things are going wild. And I know from a design perspective within Square, the whole ideology was like to get out of the way of people running their business, the point of sale and all these things should not be hindering things. They should just be folding into the background. They should be seamless because we want, I mean, business owners just want to run their business and showcase what it is that they're trying to sell or what it is that they're trying to offer. And having stuff that gets in the way of that is not helping them. It's hindering 'em. So it should be saving time and be reliable. And I'm sure with SMS, that's what you're trying to do. You're trying to save them time and be reliable.Binh Ly: Exactly. And I think a big influence on how this perspective on how to build it came from was actually visiting the customer on site. So if you go into a dental office and poke your head over the counter, they're not using Studio Display and MacBook Pros, they have square monitors back there. These are people on older machines that they don't really care about real-time refresh, but they know if they switch the focus to your browser and they refresh the page, they better come back up. That's the bar. You cannot fail at that part. So we try to ensure that every single time.Richard Moot: Totally agree on that. I mean, the amount of times that we end up looking at a redesign of a webpage and we just go, oh my gosh, this looks so amazing on this giant 4K ultra wide. And then you realize, yeah, the JavaScript payload on that is way too large and it takes forever to reload when you're on this tiny little machine that is on a very unreliable wifi connection that's not saving somebody time.Binh Ly: Not at all, the other thing that really influenced the decision to go with Rails is the rails console. Whenever something seems off, I can just SSH in and connect and start looking at the data that way. That's a really helpful tool for me. I think that's why I don't choose other tools. If they don't have that ability, I can't do it.Richard Moot: Yeah, I mean there's a power to convention over configuration. I think even as a developer, as I've evolved over time, I used to be really into building something with Express and Using React, and I love the customization of it. And as time has gone on, I'm like, I really wish there was a little bit more convention here. Everyone kind of develops their own conventions and it saves you a lot of time in your decision making because it gets you closer to solving the actual problem of you're building a business, you're building a service, that's what you want to do, and convention gets you there much more quickly.Binh Ly: Yes, for sure. And also the other hard learning was customers don't really care about your stack.Richard Moot: Yeah, no, they don’t.Binh Ly: When I first graduated university, I worked at Real Networks, the makers of the Real player and my boss there, this C++ master. And he was really into generics and C++ templates, so that was kind of my worldview. Everything has to go through the C++ pipeline, otherwise you're not a real programmer. But when you're talking to a business owner, any mention of that kind of stuff, right over their head, they don't care. They just want it to work. And so I just try to pick the thing that is the most solid that gets me there.Richard Moot: Yeah, I could only imagine going to a Square seller and be like, Hey, do you want us to build this with a serverless technology stack that has edge functions? And I'm sure they just be like, I just need to work. I just want to take a payment and make a sale. I don't care how it works.Binh Ly: Yeah, exactly. Especially with Square merchants, they have the Square on their phone, so they always say, can I use it on my phone? They don't want to have additional hardware or anything. It needs to meet them where they work primarily.Richard Moot: A hundred percent. I mean, I think I've regularly had, when talking to members in our developer community or people who are wanting to participate in our hackathons and they're building out all these web dashboards and other things, and most of the time I'm trying to encourage them understand that the Square seller spends most of their time sitting in the point of sale or sitting in the Square dashboard. That's how they're running a good chunk of their business. And so you don't want to be trying to pull them out of this. You want to find a way to be injecting or hooking into this to compliment their workflow, not pull them out of it.Binh Ly: It's also really funny with the younger workforce, maybe this is not okay to say, but in talking to the owner, they talk about some of the friction that they have with their staff. They're willing to do certain operations. I think having a mobile app for Operator is a big win because the staff is already on their phone doing whatever it is that they're doing. If they can do their job from their phone, that's one less battle you have to fight. It's a job.Richard Moot: Yeah, well, there's something also I think sometimes interesting about, so I've talked about this, people on my team, and this isn't necessarily with phones, but I feel like there's times when working from home and having multiple monitors can suddenly be very distracting. Whereas if I pull my laptop out, I have single view, I'm single focused on a task, mobile devices that actually really lend well to this. You can only really be in one app at a time. And so when you're in that app, you're focused into what you're working on there. Granted, I know we all switch between apps. I mean maybe not, but I know I switch between apps all the time on my phone, but at least when you're in there, you're focused and you're working on this particular thing. So I do think that there's a power to that to allow them to work in a constrained environment that keeps them focusedBinh Ly: A hundred percent. And recently, maybe in the last six weeks, I've discovered a super mode that I installed Linux on an old MacBook Pro, an M1 MacBook Pro, and it's like my favorite development environment now because there's nothing else on there. It's just BS code and the Kitty terminal and that's it. But you get the nice hardware feel. That was always a big barrier. I didn't want to use a think pad with this weird track pad and things like that, but.Richard Moot: Yeah, I've done that. I've done that all on a Linux build and I thought it was going to be great for me, and I ended up just quickly reverting back to my MacBook. So I think there, there's definitely a power to having the good hardware component, but then the simplicity of like, Hey, all I have here is all my development tools. There's nothing here to distract me.Binh Ly: And then the dev containers were introduced in Res-7-2. Once I learned about that, I was like, I can't imagine doing things without this.Richard Moot: Yeah, I agree. Well, I see that we are coming up on our time here, so I'm going to say that that's probably it for today. But I want to give a big thanks to Binh for joining us. Just so interesting, every time I've chatted with you to learn more about SMS, how landline SMS can work. If you could just let us know where can people go to learn more about Operator or follow up?Binh Ly: Yes, so Operator is, like we mentioned before, it's kind of white labeled and hasn't been, you can't sign up for Operator yet, but that will range in 2025 as we add sales staff to go out and evangelize some of these developments with the rest of the world. But I'm online at Binh Twitter and Instagram, just BINH. And then in 2025, you can go to Operatorapp.com to sign up, to start.Richard Moot: Alright, well I think people should definitely go and check this out. For all of those listening, if you've got a project with Square, we'd love to hear about it. Please reach out to us on Discord or on X at SquareDev and don't forget to subscribe and check out developer.square.com for more. Keep building and catch you next time.

  --------  

  40:51
• Beyond the Network: Unlocking the Power of Programmable Traffic with Ngrok

  Richard Moot: Welcome to the Square Developer Podcast. I'm your host, Richard, head of developer relations here at Square, and today I'm joined by Peter from Ngrok. Peter, thank you so much for being with us here today. I'd love for you to just give us a little bit about yourself, a little bit about Ngrok, and let's kick things off.Peter Shafton: Sure, hey, thanks Richard. Thanks for having me. So my name's Peter Shafton. I'm the CTO of Ngrok. I've been with the company for a little over three years, and I actually learned about Ngrok way back from when I was at Twilio about 13 years ago because the founder of Ngrok, Alan Shreve, was an engineer on a team there that I was running at Twilio. So I first started the voice and messaging parts of Twilio, that was actually the beginnings of Ngrok. But I've been sort of bouncing around Silicon Valley for a lot longer than that. A bunch of companies you all have heard of, probably Cisco and Yahoo, those who are old enough, SGI, and then a bunch of little startups that people maybe didn't hear of as much. But yeah, that is my past. So I'm a developer at heart, an engineer at heart, and then somehow ended up doing architecture and running technology.Richard Moot: I feel like they always trick us and lure us into this in some way. I mean, it's very alluring, but then eventually you're just like, wait, what happened? I'm now the CTO of a company.Peter Shafton: That's right. As long as they don't take away the keyboard, I can still type code there. I'm happy.Richard Moot: Do you still get to occasionally write some code for things or do you find the time for it?Peter Shafton: I do. I do. I do it badly, and most of my engineers hopefully don't let me do that. I end up in the data space more than anything or through SQL or the data warehouse and data lake areas, but at times I'll write some low level code and networking code and things like that.Richard Moot: I mean, you got to find the time for it. It is kind of invigorating, but at the same time, I would agree that outside of Dev Rel, I am less inclined to build stuff that's going to be relied on in production, and I'm more inclined on building, here's how to do this particular thing, or here's a little script for pulling some data together so that we can make sense of stuff.Peter Shafton: Yeah, I think most of us got into this because we loved writing code. I think if you'd asked me, I was a young kid, an Apple++, and I didn't realize this was a career. I just thought it was a really cool hobby to have, and then the thought that somebody would pay me to do it, and so this is the piece that got us excited. If you left most of us alone, we would still be writing code. It's just nicer when you're doing it at scale and allowing other people to see what you built versus,Richard Moot: Yeah, no, I mean I think that it's really well put. I mean, most of the time I'd build tiny little automations in my house for like, oh, now it's much easier that when I want to go to bed, it turns all my lights off at one time instead of just me having to go through all of them. But now it's like now I can solve problems for thousands of people, millions people.Peter Shafton: Oh yeah.Richard Moot: So you used Ngrok a long time ago before you actually even joined the company. I'm curious, what was it when you first used, well, two things I kind of want to answer. For anyone who's maybe not familiar, we can give a quick little explanation of what Ngrok is and then what was it like when you first used Ngrok that made you fall in love with it, or I don't know if you'd call it love, but I mean I felt like I fell in love with it when I first used it.Peter Shafton: No, it's a good story. It's a good story. Yeah. So there was an engineer by the name of Jeff Program. He was one of the early engineers at Twilio at the time. He had come from NASA and he built a tool called Local Tunnel, and he built it for a very specific purpose. So Ngrok basically fit into the webhook infrastructure that was Twilio. So the early days of Twilio, it's still true today is all powered by webhooks, which effectively means you get an inbound phone call, you get inbound text messages, you want to respond to that and control the programmability. You had to respond with XML, with twiML at the time to an inbound web request because that looked very much like how the internet worked. They figured people were writing PHP scripts or Python code for a website. They were used to getting a form post and responding, they thought, you can just reuse that infrastructure. It just happens. It's a telephone that's calling you instead of a browser client making the request, which made a lot of sense because that was what the developers were doing and it was easy to build on.The challenge was if you're inside Twilio and you're building the product and you need to be able to test it or even iterate on what you're doing, a very slow loop is I build a thing, I deploy it to production, I let a webhook hit it and see if it works or not. What we wanted to be able to do was change the code, maybe even sit in a debugger and have the webhook come in and trigger our code. And so to solve that problem, Jeff built something called Local Tunnel, and it was a simple bit of Ruby code that effectively gave you a dynamic URL.So if you think about it, they call it reverse proxy, but you basically have a little sidecar sitting on your machine. It makes a network connection out to a server that is on the public internet and it tells that server a domain name to use, so fubar or whatever it is. If you hit that request, it'll basically tunnel the traffic back to your local machine so you can iterate and have something that kind of looks like it's on the internet, it's accessible on the internet, but it is not otherwise there. It's on your local machine and if you shut it off. So that was local tunnel and it was used inside Twilio. Alan was one of the engineers at Twilio using this, and it was fine when it was just a few of us. As it scaled up, we realized the Ruby implementation was not particularly scalable, didn't really serve our purpose at scale. And so he said, I think I can rewrite this and I think I could do it much better. And he started to iterate on it and go, and when he left Twilio, he said, I'm just going to make this a side project. It was, but it turned out it was a side project that went very, very far. So the early implementation of Ngrok was very much a tool for testing, for testing webhooks, for developing websites, developing APIs, and that was kind of the early history of the product and of the company.Richard Moot: And that's exactly where I probably first started interacting with it when it became this sort of tool that was put out there. And I remember even really early on, it was very, I mean, it was straightforward for signing up if you actually wanted to use it beyond the sort of free version of it. But I got to say that after just installing that CLI tool, putting it in your local bin folder and then just calling it from your path wherever you want. And I think the first time that really got me was when I was working on something at a company and I was trying to update stuff on a marketing page and trying to get whatever it is that they wanted, the way that they wanted it. And I just ran Ngrok and had a URL, and then I was able to send it to them and say like, Hey, is this what you are wanting it to look like? And they were just immediately giving me feedback and testing things and I could see all the traffic of what they're doing in there. And that was a really amazing thing because normally I would've had to take all of that, push it somewhere, put it up on a staging version, deploy it somewhere, and I really was not at that point, I was like, I'm just trying to move things from one place to another and I want to show you what it looks like on my machine. The ease of using it was just so great.Peter Shafton: Yeah, no, it made a lot of sense. I think in the early days it still does for different reasons, but the early days, sort of the aha moment, we have a tagline now we use that says online in one line, which basically meant you could download the Ngrok executable type in a simple command line like you might for Curl or something and you'd be up and running. And I think the fun part was both testing for folks like you and I iterating on code turned out, and Richard and I were talking a little bit before this about having automated home infrastructure and those of you who have put up stuff in your house and automated it and realized, well, if I want this to be accessible when I'm not at home, then I need to somehow punch a hole into my home network.And that often meant messing with the settings on an Xfinity router or AT&T or something else to figure out how do I open this thing up and allow access. And Ngrok made that really simple too. And so we found a lot of developers that were hobbyists and that were doing and automating things. And that sort of device gateway or access became the early days of how it was used.I think the scary part is once you put something on the internet, you're like, okay, this is out there. Then you forget that it's on the internet and everybody can hit it and use it. And so having other sets of capabilities that do allow you to put authentication in front of it, all of these became the obvious kind of next steps. And if you look at the history of the company, you can see where that progress happened. But it was always that sort of aha moment that I think got early developers and I mean, it's crazy sort of the scale of this hit. There are over 10 million developers that are using Ngrok and thousands every day that come down, which you think of somebody that was bootstrapped, it was hard to imagine that you would start that way.Richard Moot: Yeah, no, I mean the growth in the evolution of the product has really impressed me over the last few years because initially it just seemed like, oh, it seems like such a simple straightforward tool. And when you're a developer just trying to say like, oh, I'm just trying to build and test my webhooks and I have to expose those to the platform that's going to call them if I want to do any kind of local testing. And then you just go, okay, that's it. But then I think there's other people who just went like, well, what else can I use this for? And you really start, tell us a little bit more about how those use cases started to really evolve. Because I think it's very interesting to sort of understand, yeah, I'm punching a hole. I can do Webhook testing or I can let somebody see my local development, but this ability to communicate within the same network in a secure way is pretty common for a lot of really large apps.Peter Shafton: There's a lot of things that are really interesting, and it paralleled a lot of the story of the time I had at Twilio as well, which was if you build a platform, it was sort of built of these basic primitives, but it was programmatically driven like you had an API to manipulate it or set it up and you could scale it from anywhere. It enabled all these use cases you're alluding to. And so that is sort of the fun of what Ngrok has become and continues to become in the simple sense you're testing, but if you think about it, at some point you have to handle webhooks in production. Often you're connected to Square, you're going to handle webhooks from that API, they have to go into your infrastructure. You want to manage them and why stand up another infrastructure to do that?The benefit of being able to shut things down or keep them up or scale them. A lot of the things that we've built and continue to build echo the problems I had at Twilio, if you think about when you first put an API on the internet, you're like, well, my first problem is just making it accessible. You're like, great, I've got it up there. People can hit it. This is wonderful. And then there's sort of that uh-oh moment where you say, well, wait a second, a lot of people are hitting it and I have to probably rate limit them, right? I don't have capacity to handle all these people and I don't want to throttle my request to one customer because another customer doesn't have enough bandwidth. And so I need rate limiting. I have to rate limit by customer ID, and maybe I have some customers that pay me more and I want to give them more capacity.And at Twilio, I had to build an API gateway team. I had to build a team that could do that. And then the other problems you start to have is, well, gosh, I have customers that aren't just in my city. They're actually hitting me from all over the world. And the latency some of them are getting is quite bad because they're coming in from Asia or from Europe or from somewhere else that don't have infrastructure there. And so at Ngrok, we literally had infrastructure to support that so that you could go from just running it locally to running it remotely. But I think the fun part about having a platform like that is, and I think you alluded to this, is people just auto discover it. I'll give you an example of a story. There was, and I won't tell the name of the company just to make it more fun, there was an IT guy who works at a company that sold a food product and expected to receive mobile ordering, but they didn't have a mobile ordering infrastructure yet.What they had was a bunch of brick and mortar stores people would come into and order things like coffee. And he was tasked with trying to figure out a way to make this work, and he grabbed the Ngrok Executal and stuck it on his point of sale system that was running Windows. And lo and behold, not only did that give him an API web endpoint that he could hit, but it gave access to his database and he could look at the end of day sales for the store location by pulling down the database at night and he could push updates for discounts and product in the morning. And lo and behold, he could drive all them over. And he sent an email to Alan. He said, I just did this thing. I downloaded Ngrok, I can do this. And Alan said, sure, it wasn't what I thought about when I first built it, but it works for that.And lo and behold, that store grew to 28,000 locations and that company does millions of dollars a day of mobile ordering across Ngrok because those store locations, if you think about it, have home networks like you and I do. These are not, they often use some cases, a cellular connection. In some cases they were using AT&T or Comcast or whomever. And so he needed to be able to have a reliable network connectivity. Then he didn't want to go and update the routers on each of those locations or convince the IT team. So it was sort of a fun, interesting use case. I've seen people control smart mirrors, remote control boats and planes that they remote into and give access. And these things just have cell cards in them. They're going over the cellular network. So it's sort of fun to see both the things people have done that are hobby and interesting and unique, but also the places where people have used this and built entire businesses on top of it because connectivity was key for them.Richard Moot: Yeah, I mean, it is one of those things where it's like, I think it's like when thinking about the layers of an app, and I think it's often that the network layer is one that, I mean, I know when you're dealing with really large scale apps, you inevitably always think about the network layer. But I think when we're first building out a web application, we kind of just take the network layer for granted of just like, yeah, it's going to be exposed in some way, and then we have an auth model. But there's so much more that you can do there that, I mean, I'm just imagining that with Ngrok, that it's so much more possible to say, yeah, we have a private network that most of our application infrastructure is communicating on. And occasionally it would be great if for something that's say existing in a more hostile environment, say a storefront with their own networks that are connecting to devices, we can only trust so much.And it's like, but you have this one machine, the point of sale where you're like, I want to be able to punch a hole for just this specific device and it can talk to my private network through this particular endpoint that I've created here for it to communicate. And that is actually, not a lot of people have to deal with this, but I feel like it's one of those things, it suddenly goes, oh, this makes this so much easier than having to think about mobile device management and certificates and certificate pinning and all these other security mechanisms that you have to employ when you're dealing with all of these different devices.Peter Shafton: You're exactly right. And it's interesting because we really saw, what we have seen is about four or five use cases or traditional, the device gateway, that sort of connectivity to remote devices. An interesting one, an API gateway has been another interesting one. I'll give you an example. When I was at Twilio, we acquired two companies that a lot of people had heard of. One was called SendGrid that sent email, and another was Segment that did data processing. And imagine you acquire companies that are fairly significant. They each have significant data infrastructures. They already had a footprint in GCP or AWS or in SendGrid's case on-prem, like a private data centers in Colorado. And when each of those companies set up their infrastructure, they created private IP ranges. And when Twilio acquires them, gosh, we should have a single API, right? You should be able to send email to SendGrid via the Twilio API. And you think about that, well, that's hitting the Twilio endpoint. And you might say, well, okay, all we'll have to do is things that are slash twilio slash email should go here. How do I do that? They have their own ingress points, they have their own, and they're in a different environment. I can't just take the DNS that goes to one place and send it somewhere else. I can't split it in the middle and say things that are -V1 should go here. All the complexity of the network and the great intelligence we had ahead, we had totally overlapping private IP space.So I couldn't just route for one infrastructure to the other because the IPs were redundant that it was not clear where I was going with the traffic. And so Ngrok, if we had had it at that time, would've enabled me to basically say, Hey, listen, I want to shard this anywhere I want. The network is effectively as programmable and configurable as the other parts of my software stack are. And I could have said, Hey, my V two version of IAPI, which will be somewhere in this path, is going to route to my new infrastructure that guess what I've stood up in the Azure Cloud or things that match these headers or these parameters, I want to send this other way. And so being able to do that through a programmable network interface was really what is powerful and interesting about Ngrok. And there is an infrastructure today that we built that basically uses traffic policy.And the idea is that all of the information that's available to you on an inbound request can be used in rate limiting, in routing decisions, in decisions as whether to force somebody through authentication flows. And that allows customers of ours to solve problems we hadn't even considered today. So we see customers who are doing exactly that, sending some traffic to different infrastructures to challenge certain users with authentication or add special headers or modify their payloads either inbound or outbound based upon what they see. And one of the fun things we worked on was to scrape the web for public data sets about IP addresses. So you know, have geo-data, everybody's kind of familiar with Maxmining, and other places, well listen, given an IP, I can figure out the country. But it turns out people publish the IP addresses of TOR exit nodes, they publish the IP addresses of block lists or the ranges of IPs that belong to Square or Amazon or Microsoft for their services.And you can annotate those IPs as they come in and you could say, Hey, listen, not only am I going to validate the square authentication token and signature of their webhooks to know it came from Square, I can also tell, Hey, it came out of a Square IP address. And so it's really hard to spoof this and all of that information. Not only do we use ourselves to protect our platform, but is available to our customers to use in really interesting ways. And so that's been really fun to see. Some people are used to blocking countries, but you can imagine doing things like when a request comes in, I can look up based off the authentication, which customer that is. Are they a VIP customer? Should I send them into another infrastructure? Should I provide to them a different rate limiting rate? And that's been sort of fun to see for us. It's a platform for everybody else. It's a playground.Richard Moot: Yeah, no, totally. And I mean, I feel like one thing that is interesting about what you've sort of touched on in there that when I first joined Square seven years ago, one of the first things I did was I was immediately starting to build something that's going to be making use of our webhooks. And as my knee jerk reaction was to use Ngrok to expose an endpoint and then register that and then start seeing my feed of events happening in the system. And one of the things that I think most people might not understand at first, because when you first say, Hey, we're going to punch a hole in the network and it's going to be publicly available, you're exposing something onto the internet. I think that the reality is it's more secure than you might think. It's, and I'd love to touch on the ways in which Ngrok is sort of, I don't know if I'm stretching and saying secure by default, but I think what ways is it sort of secure by default? So when people use it, they're not necessarily worried like, oh my gosh, somebody's going to be able to call into this and start seeing stuff on my system.Peter Shafton: Yeah, there's a bunch of different levers you have. So you think about it, and it's interesting because it kind of depends on people's, you see in the old days or the first early days, you could have an HTP or TCP endpoint, which kind of meant you could basically have an IP domain name and a port that was open to everybody and well, that's very powerful. If you're fronting a thing that you just want to get online, we already know about HTPS and TLS and SSL as a mechanism to know at least you're talking to the thing you think you're talking to, right? And part of that is, is there a certificate for it? Is that certificate up to date and valid? And if you've been running network infrastructures, sometimes your certificates expire and you have to renew them Ngrok takes over all of that part of the management, right?The second thing you can think about is the traffic transiting the Ngrok network secure in another way, sorry, secured through TLS, but where is TLS terminated, right? Do we terminate on our edge? Does it need to be terminated all the way back at the service that's actually handling it? And at Ngrok, you can choose where you want. So we have customers that do zero trust, that basically we don't see their certificates, we do not terminate the traffic, we route it through and it's up to them to terminate. But in the world where, and there's a bunch of protections you can do with that.One example is you might know the IP addresses, these requests are coming through and you can whitelist those and say, Hey, listen, these are only IPs I'm going to allow or there's a set I want to deny. And with the IP intelligence, you can effectively say only from this ASN or other pieces that make it even more flexible. There's DDoS protection if you think about it. It's like, okay, put this thing up. Am I going to get hit with a packet flood? And so we have a firewall layer that does detection of SYN packets and it does it globally. So it has to basically know, okay, maybe it's an attack that's actually being spread wide enough across the world that any single node doesn't kind of see it, but in aggregate I do. And so that is a piece that we have both to protect ourselves and to protect our customers.The other use case, we talked a little bit about authentication, you can turn on authentication and say as an example, Hey, I want to use Google Auth and they have to match this domain as an example. We use Google Auth for Gmail and other pieces. And so if you don't come in with an Ngrok.com domain and authenticate through Google, I don't let you in, but you can imagine you can also use a subset of emails within that domain. Maybe it's just mine, maybe it's just me and you, Richard. And so that's another way. And then when you're doing machine to machine communication, obviously the machines are not going to authenticate through that, but they can do mutual TLS and in that case they have a certificate that they have to mint on the client side and validate that that's coming in. And so all those are the building blocks that kind of allow you to secure the edge that you have.And then the last piece of it kind of becomes observability. And so if you've used Ngrok in the early days, you kind of remembered you could hit port 40-40 on a local port and you'd have this browser window and the browser window showed you, Hey, this is the traffic. Not only could you see it, but you could actually kind of replay it. So as you alluded to, somebody was testing a webhook, you're like, great, my code is broken. Can I just replay that? And you could, we've moved effectively all of that logic server side. So now if there's a team of developers, you can see all of the traffic that is coming through, you have the IP intelligence on that data too, and you can kill sessions and block sessions. So if you're doing authentication through us, you can say, no, I'm going to take Richard's token away.And any track that he had will cease to be functioning right now. And I can see the history of what he had done. And all of that was pieces that we thought were really interesting. And then conceptually, you can replay through that same interface. And all of these were built effect to solve the problems Alan and I and others we're seeing in our history over the last 30 years. But the stuff that developers are coming to us and telling us like, Hey, I'm seeing this. I need you to be able to do this. I need you to be able to scale this way. That's really the power of the platform.Richard Moot: No, I mean it's really impressive. I mean, there was several years ago when we had acquired Weebly, and as part of trying to get them integrated into the rest of our infrastructure is one of the primary ways that they would communicate was through our own APIs. So as you could imagine, one of the things that most of the engineers working on that integration process was having to integrate with our webhooks and the go-to thing was like, yeah, you spin up Ngrok. And then once we were able to actually get it working within our networks, it was a more secure way for them to be able to just spin up, start seeing feeds and building into our systems to sync databases between Weebly and Square online. But yeah, so when Weebly was onboarded, I mean one of the things that we pushed really hard for was having better tools for doing these integration processes.And it is really great to finally be able to use this more predominantly and also recommend it to people coming onto our platform externally. So third party developers who are like, Hey, I really want to build an app on the Square developer platform, our top recommendation is usually to just use something like Ngrok to be like, if you run this, you can start building against our webhooks. I know previously we've all probably used, what was the name of that site, I want to say it was website.webhooks or webhook.website, or it was basically,Peter Shafton: WebHooks.fyiRichard Moot: Yes, where you would just can drop it in and you can see events come through. But the problem that I've always had with that is, yes, I can see the events coming in, but it's not running against my code. My code is not reacting to any of the webhook events. And so I'm not seeing the additional parts of this of actually seeing is my code that relies on the Webhook events working appropriately. With that, I also wanted to just plug one other thing that I'm excited to try out. I will be forthright in that I have not tried it out, but the SDKs that you have for actually using Ngrok in application code, which I will admit it kind of hurts my brain a little bit to think, okay, so the code itself is not going to be aware that it's actually really tunneling out traffic and responding to stuff through here. But tell us a little bit more about these wrappers that you have to enable that kind of integration directly within somebody's code.Peter Shafton: Yeah, there's two fun pieces here. So the first is you alluded to, most people who are familiar with Ngrok or have used it in the last decade are used to downloading an executable. You're like, Hey, did you download Ngork? And that's this traditional thing. It's a separate executable startup. For those of us developers that might not be a big deal if it turns out you're building with Ngrok and you want it to be part of the infrastructure, you're standing up sometimes you don't want your customer, maybe as you all alluded to, you're building a Square app. You don't want to tell your customers, Hey, go download Ngrok to use this. And so we built SDKs in a bunch of different languages, Rust and Java and JavaScript and Go and Python that you can just build the capability. And what Richard is alluding to is we did it in a way that effectively in most of your code, you're used to standing up like an HTP server or something like that, and those servers are often based on basically a TCP socket, a listening socket. Effectively they open Ngrok for all intents and purposes, looks like a listening socket. So effectively these libraries provide a socket that just can be handed to the HTP server objects in Java and Python and Flask, Restful and all these things that the framers you're used to playing with. And so your code isn't aware and conceptually could choose on the fly to be Ngrok enabled or not. And the interesting part about that is often when you're running locally, you have Ngrok there because otherwise you're not accessibly can't get requests from the outside world. And when you push to production, the incentive is like, well, now I'm re-architect the thing. I'm going to put engine X or some other form of ingress in front of it and talk to the IT team to get it all set up.The answer is, if you built it this way, you don't have to do that effectively. You can just push the executable up and it is actually internet enabled at that moment because it is an outbound socket that's powering it. And so that's really cool for the SDKs and the use cases that people ship it. There is another part of that story that I think is really fun and interesting is a PCLC called the Ngrok Kubernetes Operator. So in this world, you basically have a helm component that you install, and there effectively is a service within your Kubernetes cluster that acts as the ingress to that pod. And so when people are used to using Kubernetes and using the Gateway API or setting up an ingress or load balancer, they basically leverage Ngrok to do that load balancer. So now you don't have to worry about, is my Kubernetes infrastructure running in AWS and I should use ALBs and NLBs, is it running in Azure or GCP?Is it running on my local machine? So you can basically have ingress to services and the same definition of endpoints. The other fun part about that mechanism is you can effectively tunnel traffic from your local machine into a Kubernetes cluster. So imagine you have a Kubernetes cluster running in production or in staging and it has a bunch of services running, but one of them you want to iterate on that workflow normally would look like I work on my code, I write the Docker file, I deploy it to that cluster, and then I test it and realize I broke it again. In this world, effectively that service endpoint that's in that cluster just tunnels the traffic through to your local machine. So it's sort of how you were using Ngrok today to say, my service will be visible on the public internet. Now you can say my service would be visible in a Kubernetes cluster, and that allows you to do things what we call private or internal endpoint.So if you think about, Hey Ngrok, I put this thing up, it's on the public internet, you can actually say, Hey, Ngrok, I put this thing up, but it's actually not publicly accessible. It's only accessible from within this Kubernetes cluster and other services and pods within the cluster can hit it, but people from the internet cannot. And if I want to expose my local machine, then we manage the mutual TLS certificates that guarantee that the traffic coming into hit your machine is actually coming from that cluster. And so those pieces are really interesting building blocks as well because we see a lot of people who, their deployment model actually is Kubernetes, right? They're actually deploying pods, they actually workflow, and this works much more elegantly for that use case. And so that's a piece that's been fun as well, and it works in a similar fashion to how you think those SDKs work. And guess what? It's actually using our Go SDK, right, so it is both a consumer and Ngrok. The fun part is we dogfood everything. So our API, our website, our docs pages are all fronted by Ngrok. So they are really limited and protected and globally redundant by our own platform. It's a little meta if you think about Ngrok's behind Ngrok, but it is.Richard Moot: But I mean, I find it hard to find a better way to really iterate because on a product, because you're customer zero constantly. And so if it works, it's working for you, it hopefully is working for everyone else and you can continue to improve and expand, but you're still serving customer zero, so you're not breaking yourself otherwise you're going to be in a lot of trouble.Peter Shafton: We had a funny story. So we put the website, we had a lot of the components behind Ngrok already, the API and the dashboard. We hadn't yet put the website Ngrok.com. And when we did that, four hours later we got hit by a DDoS attack and we were like, okay, this unexpected. At first we thought we had done something wrong in the deployment. Turned out, no, we were getting hit and we looked in our observed, really realized the traffic was coming a lot from block lists, a lot from Russia and China. And so we said, well, okay, we flipped on a bunch of the traffic policy protections to block tour exit nodes, to block things on firehole or other block lists and Russia and China explicitly. Four hours later we got hit by another DDoS attack. And lo and behold, at that point, it had no effect on our infrastructure.So it was sort of fun to see how quickly we got inside and I would say it was about 7,000 times the amount of normal traffic we see on the website. So it was clear as much as I know Ngrok is popular and I love to see spikes of traffic, that was not something that we expected, but it was a funny use case. Normally that would've taken days to figure out how do we thwart this, how do we spin up enough infrastructure to protect it? But the pieces were there, we just had to turn the ball.Richard Moot: Yeah, no, I mean that's amazing that you can be able to react so quickly and make those changes on the fly like that and then just immediately see a result of like, oh yeah, we see the next wave coming and suddenly we're not affected anymore.Peter Shafton: Yeah, no, it was fun. It was cool. I mean, as much as it's not fun getting attacked, that was fun.Richard Moot: Yeah, I mean I think with time we can realize it's more fun, I'm sure in the moment it's like, no, we got to get this problem solved and we got to get it solved fast. So I think the final thing I want to touch on is sort of switching gears a little bit, as succinctly as I think we can try to do it with the time that we have remaining, is I'm curious how things have changed for you over time as you've gone through the years of as an IC engineer moving more into architecture and now as the CTO, how has that evolution sort of changed for you in terms of how you approach engineering problems? And tell us a little bit about that.Peter Shafton: Yeah, it's a good call. I mean, I think as an IC, you're sort of in learning mode. Everything you're doing is your first time maybe, and your goal is to absorb from everybody around you. There's no illusion that you yourself are going to build everything. You're just going to learn from the people around you. And I think as I've grown, that's what I've done both learn how do you grow an organization and how do you grow an infrastructure and what kind of things, what is important when in a lifecycle? When I joined Twilio, there were 20 people, and when I left in 2022, there were 8,500. And so there was that arc that was how does the org grow over time? But it is also what decisions do you make as you're building the infrastructure? And I think those are the pieces you learn. How do I build an organization?What kind of engineers, what skillset sets should they have early on? What do you want to tell them to optimize for? When you're building a product like Twilio or even Ngrok, the reliability at scale is very, very important. I remember engineers coming to me and saying, I want to build this in Ruby, it'll be much faster for me to execute on it. And I had to tell them at Twilio, you can't build in Ruby because this has to scale for hundreds of thousands of phone calls, calls we build in Java or C, Go so that our customers can write in Ruby. And so there was that part of a learning of which, but that's not always true, right. There are some things that's better to just build quickly in a scripting language because you don't yet know if it's a good idea. You don't yet know if it's going to work, and so Ngrok has been interesting. For me, it's a little bit of a chance to step back into technology and into things with the foresight of what will happen five years out and seven years out, what would it look like if our traffic quadrupled or we land a single customer that's bigger than all our customers. And I think those are the learnings that allow you to make some decisions that you know you can revisit. You'll hear people use the word two-way doors, and if it's a thing you can kind of back your way out of in a reasonable amount of time, then don't wait and think about it a bunch. Just do it.Richard Moot: Yeah, just go.Peter Shafton: And if it's a thing that like, okay, this is going to be a really expensive decision if I'm wrong later, those are the ones you sit and haggle about because as an engineer, especially building a company, time is the only enemy you want to move quickly.You want to try things out, you want customers to get, they love that, right? That sense of, and I see this at Ngrok and I saw it at Twilio. Jeff Lawson, who was the CEO, used to joke. He's like, I want to be the train that's a mile ahead and pulling away. And that is very much true of a startup. You want to be building features and capabilities and listening to your customers iterating quickly. And so as a CTO, my job is to keep priorities front and center to the engineers and let them know this is a thing you can do fast and this is a thing you gotta think about, but that part is fun.Richard Moot: Yeah, I think one thing, and tell me if this sounds true to you, is as time goes on, you start as an IC, you're focused on the engineering problems of what code should I write, what language should I use? And then as you start going further and further in the architecture, it's like, oh, well what services and what should the architecture of this be? But I feel like there's this moment where you start realizing the biggest problem to solve in this equation is the human element. And you have to start figuring out how do I actually structure teams so that they work together better and organize them in ways that they can focus on the most important problems? I feel like sometimes people can get really hung up on like, oh, but if I write it in this way, I can make it work in so many other ways.And I think you touched on it right there of saying, well, we can't write code and Ruby because as we go through the abstractions, it's going to be too slow when we have a customer down over there writing stuff in Go. And so we need to be in the fast, reliable, low level type thing because somebody else is going to abstract on top of it. And that requires that human element of helping people to understand there's a bigger picture and there's layers to this whole thing. And how do I organize people to make them come together in the right way?Peter Shafton: I mean, you're very much right. People love to think of software as this technical endeavor that you can use AI and build your way out of that humans will no longer at some point be necessary. And the reality is these systems are still very much human systems. And so you have both the learnings of the people you hire, which experiences did they have that you didn't in technologies or solving related problems, how do they interact with each other? Who is listening, who who's interacting, and what gets them excited? An engineer's going to do a good job if they're excited about the problem they're solving. They like the space they're working in, they feel like they're doing something that they themselves would've used or leveraged. So it's fun to work on developer tools. Your customer is probably your friend or even you, which is ironic. And then the second motivation is do I feel like I have freedom to kind of solve within some confine as an engineer myself, there's an instant ability for me to look at something and say, I know how you should write that, but I know as a leader, I can't do that because that's disempowering to engineers.And so they need to be able to be given the context of what has maybe a suggestion of where I've seen walls historically myself, but then the freedom for them to think and solve the problem in their own way. And that's what builds really powerful teams as leaders, as ironic, it sounds your job is to make yourself obsolete and unnecessary, and that's where it has to be. Allowing smart people to run and grow allows me to step out of half of those problems, right?Richard Moot: Yeah, I think what you said resonates. It just resonates so much with me because I had an engineering manager that I worked with previously, and I had discovered this architectural problem that I realized it was like we were seeing reflections of it in different issues that were arising, and I finally was able to go like, no, this is the root cause of why we're having this problem, and it was this inherent design problem. We're just actually designing these models in the wrong way, and I figured it all out. I wrote it all up, and I even had several proposed solutions, and one of the things that he said to me, which is what you just said right now, is that even though I figured out the whole problem, I'm not going to win people over or get anybody wanting to work on it if I tell them the solution that they need to go implement.I just need to present the problem as well as possible so that a team gets to go and grasp the problem, see things that I didn't see, but the joy as an engineer is coming up with the solution, and that's how you empower and allow people to grow is that you might see like, hey, there's this huge problem over here. I'm not going to solve it, but I am going to figure out what the problem is and put the right people on it, and then they're going to go figure out how to solve it. That was a huge unlock to me of it is not enough to just be right. It's not enough to just find, oh, there's this thing wrong. You have to figure out how am I going to rally people together to believe that we should address this problem and get them to want to solve the problem and show why it's going to be valuable. Those are all the human part of engineering that it's a form of problem solving, but it's not with code.Peter Shafton: Yeah, this is the hardest part about going through the engineering ranks. You miss the days where you were coding the solution. You have to be okay with that. You have to get your satisfaction from some other place. It's not going to be you writing all the code and solving all the problems yourself. That is not how you grow.Richard Moot: Unfortunately. Well, and I've actually, I've heard, I don't remember where I came across this, but it was an interesting take on why the problems that you solve actually never get easier. Because as you get better and better at solving these problems, it's only the most difficult problems. As you've grown as a leader, you go further up and you're zooming further out, and the problems that eventually make their way to you always feel like the hardest problem. And it's like, well, for good reason because you've built these teams that are going to solve the easier problems. So only the hardest problems are coming up to you, and those are the ones that you're having to solve.Peter Shafton:No, you're exactly right. It's funny, but true.Richard Moot: Yeah. It's one of those things where it kind of hurts because you go, it's never going to get easier. It's actually supposed to just continually be just as hard or harder as it goes on.Peter Shafton: Yeah, somebody told me as a CEO, you only get presented the hardest problems at the company. You have to be okay with that.Richard Moot: So I see that we're pretty much up on time here. I want to thank you again, Peter, for coming here teaching us a little bit about Ngrok. I'm super, super excited to go and try out the TypeScript SDK, and my dream is to actually put this into a CLI that somebody can use and immediately spin up test their webhooks with Square. If people are interested in learning more about Ngrok, what is the best way for them to actually go and check it out and try it?Peter Shafton: Certainly, Ngrok.com is easy. You will find our docs. You can just search for Ngrok SDK if you want, pick the library of language you want. They're available on GitHub. So our GitHub repo under Ngrok also has all of this code. These are open source, so you can grab 'em and look at 'em if you want, but pull 'em down and play with 'em and give us feedback, right? They've been out there, they're heavily used by a bunch of folks, but it doesn't mean there aren't places to improve and make them better. So definitely reach out, let us know, and we have support at Ngrok.com if you run into challenges and you need help, and I might even be manning that support some of those days. So you could talk to me.Richard Moot: Excellent. Well, you might be hearing from me. If you've got a project with Square, we'd love to hear about it, reach out on Discord or X at SquareDev, don't forget to subscribe. Check out Develope.squareup.com for more. Keep building and we'll catch you next time.

  --------  

  44:24
• Codename Goose - Your Next Open Source AI Agent

  Richard Moot: Hello and welcome to another episode of the Square Developer Podcast. I'm your host, Richard Moot, head of developer relations here at Square. And today I'm joined by my fellow developer relations engineer, Rizel, who's over working on Block Open Source. Hi Rizel, welcome to the podcast.Rizel Scarlett: Hey, Richard. Thanks for having me. And I know it's so cool. We're like coworkers, but on different teamsRichard Moot: And you get to work on some of the, I'll admit I'm a little bit jealous. You get to work on some of the cool open source stuff, but I still get to poke around in there occasionally. But today we wanted to talk about one of our most recent releases is Goose, and I would like you to do the honors of, give us the quick pitch. What is Goose?Rizel Scarlett: Goose is an on machine AI agent and it's open source. So when I say on machine, it's local. Unlike a lot of other AI tools that you use via the cloud, you have everything stored on your computer, private, you have control over the data, and you get to interact with different lms. You can choose whichever you want, whether it's GPT, sonnet, 3.5, whatever you prefer, you get to bring it.Richard Moot: Awesome. And so I'm going to hopefully give a little bit more because I want to just kind of clarify for Square developers who might be coming in, they're like, they're just building other APIs, SDKs, trying to extend stuff for square sellers. So when we're talking about an agent, an agent, I always end up thinking the matrix, the agents and the matrix. And from what I understand, it's not too far off. You give it instructions and it will actually go and do things on your machine for you write two files, edit files, run commands. It's almost like doing things that a person could do on your computer for you.Rizel Scarlett: Yes, exactly. That's a really good description. It doesn't just edit code for you. It can control your system. So I had it dimmed, the lights on my computer open different applications. You can really just automate anything even if you didn't know how to code.Richard Moot: Yeah, I mean that's one of the things that I didn't even really think about when I first tried Goose. So one of the fun benefits of working here at Block is that I got to have fun with it before it actually went live. And one thing that I didn't really think about until I tried the desktop client and I forgot to allow the plug, there's two different ways you can interact with it. There's the CLI and the terminal, and then there's a desktop client, which I think right now works on Mac os.Rizel Scarlett: Yes,Richard Moot: I know there's big requests and to have it work in more than just Windows.Rizel Scarlett: Yeah. Yeah. Right now, I mean we do have what I think is a working version of Windows, but the experience for the build time is not great. So we're still working through that.Richard Moot: Yeah, well, having my own wrestling with working with the Windows sub Linux, I only really think of it as WSL. I've had so many headaches of trying to deal with networking and connecting and when do I need to switch to the power show versus a terminal, and it's all the reason I end up falling back to doing all of my development on my Mac.Rizel Scarlett: Yeah. I haven't used the Windows computer since I was an IT support person. I don't even know what the new developments are now.Richard Moot: Yeah, I mean I recently got burned by that where I didn't realize that in order to do certain virtualization stuff, you had to have a specific version of Windows, like some professional version, and then that enabled virtualization to run a VM of something interesting.I think since then they've baked in the Windows sub Linux thing, which is basically just running Ubuntu in a virtualization for you. But that was an eyeopener, but thankfully Microsoft's working on fixing these things, but we digress. So coming back to Goose and what is it that most people have that you've sort of seen from the community as they've been starting to try it out and use Goose?Rizel Scarlett: Yeah, I mean I just see people, well, a lot of it is mainly developers. That's the larger side of just using it to automate a lot of the tasks that they are doing. Maybe setting up, what am I trying to say, the boilerplate for their code or just sometimes other different things. I see people wanting to build local models and just in general or doing things with their kids, but I've also seen people doing silly experiments. This is where I find a lot of fun where people are having Goose talk to Goose or having a team of different, I guess geese, a team of agents and they're basically running a whole bunch of stuff. So they had one Goose be the PM and it was instructing all the engineer agents to perform different tasks. So it's a varied amount of things, but a lot of people are just trying to make their lives easier and have Goose do the mundane task in the background while they do the creative things. I've just been doing fun silly stuff. Like I had Goose play tic-tac-toe with me just for fun. I just wanted to see if it could do that and that was cool. Yeah.Richard Moot: Have you beat it yet?Rizel Scarlett: Every time I'm disappointed.Richard Moot: You think it'd be way more advanced? I mean tech to can kind of, if I'm not mistaken, I think based on who goes first, it can be a determined game as long as you play with perfect strategy.Rizel Scarlett: Yeah, I told it to play competitively. I'm still working on the perfect prompt. You always let me win Goose what's going on.Richard Moot: Maybe that's part of the underlying LLMs is that they want to be helpful and so they think they're being helpful by letting you win, otherwise you wouldn't have fun.Rizel Scarlett: That's true.Richard Moot: Well, one of the things I was very fascinated by when first trying out the desktop client versus the CLI, because I habitually used the CLI version, but when I first opened up the desktop client, I had asked, what is it that you can do? And one of the things that it suggested that never even occurred to me was using Apple Scripts to run certain automations on your system. And I immediately just went, okay, can you organize my downloads folder and put everything? And it just immediately put everything in organized folders. And that's something I used to, I mean years ago, write my own quick little scripts to be like, oh, I need to move all these CSVs into someplace and PDFs. And it just immediately did it for me and it was just, that was amazing because now I can actually find where the certain things are.Rizel Scarlett: That's so awesome. Yeah, I think you might've been using the computer controller extension, and that might be my favorite so far just because of, oh my gosh, it could actually, it's not just writing code for me. I'm like, okay, cool. There's other stuff that can do that Cursor does that as well, but it can tap into my computer system if I give it permission and move things around. I did a computer controller extension tutorial and I was just making it do silly stuff. Like I mentioned, it dimmed my computer screen. It opened up Safari and found classical music to play it, did some research on AI agents for me and put it in A CSV and then it turned back on the lights. It's so cool. I can just tell it, go do my own work for me and I'll it back.Richard Moot: Yeah, that's great. And so you touched on something that I think is kind of an interesting part about it, and I feel like I want to come back to the part to really emphasize GOOSE is an open source project, and so it allows you to attach all of those various LLMs to sort of power the experience. But what you just touched on there is the extensions. So the way that it can do these things, could you tell us a little bit about what are extensions and how are they used by either Goose or the LLM? What is the relationship there?Rizel Scarlett: Yeah, so extensions are basically, I guess you can think of it as extending it to different applications or different use cases. And we're doing that through a protocol called the Model Context Protocol, which Anthropic and us have been partnering on. And basically it allows any AI agent to kind of have access to different data. So for example, there's a Figma MCP or a model context protocol, and you can connect GOOSE to that MCP and tell it, Hey, here's some designs that I have, and Goose will be able to look at those and copy it rather than when you're maybe working with something like chat GBT, you have to go and give it context and be like, Hey, chat GBT, I'm working on this. Here's how this goes. And it takes up a lot of time. It'll just jump right in. And like you were saying, it's open source, so anybody can make MCP, you can connect it to any MCP out there that, I mean, some of them have to be honest, some CPS that are out there since it's open source, they don't all work, but the ones that do, you can connect it to Goose.Richard Moot: Yeah. And so that's kind of like what you were originally talking about, the computer controller one.Richard Moot: I'm going to hopefully describe this in a way that can make this visual for those that are listening in. But when you're using GOOSE in the terminal, when you first ever install it, it'll run you through a configuration of, Hey, it's basically setting up your profile and it says, which LLM do you want to connect to? And then you can kind of select from there and then it'll say, give me your credentials. And then after that you can get the option to, well actually maybe I'm jumping the gun here. I think it just gets you through storing that. And then you can have the option of once Goose is configured, you can toggle on certain extensions, extensions that are included, and then there's a process to actually go find these other ones that are published elsewhere and then add them in, right?Rizel Scarlett: Yes, that's correct. We have your built-in extensions, like the developer extension, computer controller memory, and then you have the option to reach out to other extensions or even build your own custom extension and plug it in as well.Richard Moot: Gotcha. And so the one I know that is the key one that's included with GOOSE is the developer extension. That's what does all the basic developer actions that you would think of, and then computer controller, that's kind of the one for doing more. Maybe tell me how is computer controller different than developer?Rizel Scarlett: Yeah, so developer extension, it has the ability to run, shell command Shells scripts, so it'll go ahead and you say, create this file. It'll say touch create this file. It'll add the code for you in the places that it needs to. Whereas the computer controller, the intention of that is that it's supposed to be able to scrape the web, do different web interactions, be able to control your system, and then this is all automating things or even do automation scripts like you had mentioned before. These are all automating things for people who may not feel as comfortable coding, but they want to automate things within their system. That's the intention of the computer controller.Richard Moot: Gotcha. And I'm curious, as this has been out there and having two different versions, I don't really want to say two different versions, two different ways of interacting with Goose with the desktop and the CLI, the desktop is really great for those that might not be more comfortable opening up a terminal. Have you seen folks coming in who are maybe less technical, who've been trying to actually use it through this way?Richard Moot: Just curious all the various types of people that have been coming in and adopting or playing around with Goose.Rizel Scarlett: Yeah. Well first side note, even though I'm comfortable with the terminal, I like using the desktop. I just think it looks more visually appealing for me. But I have seen people in Discord, I think there was a set of health professionals that they were part of a hackathon and they were using GOOSE to build whatever their submissions were. I don't know exactly what, but I thought that was interesting that they're going to build tools and submit to a hackathon even though they're not solely software engineers. So that's one example.Richard Moot: Interesting. So it's been really interesting seeing all of the different ways that people have building on it. And I mean it's been pretty exciting seeing how people have really started to just start using it. One of the things I thought was interesting was it seemed like initially some people just didn't quite understand, and I'm sure there's just work to be done in general, not just for us, but for people trying to use agents that I think a lot of people have assumed initially like, oh, where's the LLM? Why is there no thing bundled with this? I feel like I can't do anything. And we're like, no, you have to connect it to something else. But the one that got me the most interested was trying to get it to talk to a local LLM. And so I've tinkered with this over the past couple of weekends of running a llama, getting a model running. But I will admit that I hit my own endpoint where I was like, okay, I have an LLM running, but it doesn't really work with the tool calling.I think that's something maybe we talk a little bit, what is it tool calling is like that thing that how it uses the extensions. But maybe you can tell us a little bit about what is tool calling and why is it so important?Rizel Scarlett: Yeah, I mean a lot of things you said that I want to touch on. First off agents, I think from not understanding how Goose will work, I think agents are still a fairly new concept and everybody is saying, oh, this is what an agent is, and they all have their different definitions. So when I first used Goose as well, I was a little bit confused. I was like, what is this supposed to do? I think similarly when Devin came out, people were like, this is not working how I thought it would. So that just happens. But yes, open source using Goose with an open source LLM is so powerful because Goose is an open source local AI agent, and then you have the local LLMs that you can leverage it with. So you can own your data and you don't even need internet to have the LLMs running.Rizel Scarlett: But it is difficult. And like you said, tool calling. I am excited about this. I just came off of livestream with an engineer from alama. First off, the way he described tool calling was interesting. He said, it's not how I thought of it, but he was saying it's kind of how the LLM learns what it should or can do. So it's kind of like, oh, I have these set of tools here. Which one should I use for what I'm going to do? So let's say you told somebody I want to look up, or I want to go on a flight to, I don't know, Istanbul, I don't know why I picked that. What flights are available, how long will it take me? So thenRizel Scarlett: An agent will be like, oh, tools do I have? And it might say, oh, I have a find flights tool and I have a MAP tool and I have this. So in order to find the flight, it might use that flights tool and in order to figure out the distance, it might use the MAPS tool or something like that. So that's kind of how it would work and it, I think it refines the results that it would have rather than looking at all these different things, it's like, okay, I'm going to use this particular tool and get this particular output. I learned a lot about open source models working with Goose or any agent, you have to, it's a lot of different prompting tips. First off, it's best probably to ask the open source LLM what access or what tools do I have access to? Because Open Source L LMS are much smaller, so they have a smaller context window and they're not able to interact with an agent like the cloud ones. They have so much more larger content with those, so they're able to take in more memory and stuff like that. So it's like I only have this amount, so let's get to what we need to do. Show me what tools are available. I'll grab that particular tool that's needed. And then another suggestion for when building an agent, and I think Goose will probably go in this direction to help improve the experience of working with open source. LLMs is having structured output. So the structured output would tell it kind of what it can and can't do and how the format of it would be printed out.Richard Moot: Interesting.Rizel Scarlett: I know I said a lot.Richard Moot: No, no, no, that was great because it had me wondering with certain, when I started messing with one of the open source models and then I was trying to use it, I think the open source model I found was from somebody within Block who actually tried to fine tune a version of Deep Seek to be like, oh yeah, this one will work with tool calling. But then I think I was realizing I still needed probably an extension for it to actually make use of it because, and I think that's the part where maybe I misunderstood how these things work, but I'm sure that there's things that Goose does that it maybe tells the LLM almost pretext that is sent in the context. So before you even write your prompt, it has things that it will sort of give to be like, Hey, you have tools available to you or there's these tools. And so you might not see that in the terminal or in the desktop, but it's actually sort of adding these things at the beginning or maybe the end of the context to say, Hey, here's some tools available to you. Make sure that you use them. I'm oversimplifying, I'm sure, but that's kind of how I'm guessing that that might work.Rizel Scarlett: That's how I seen it. So I looked at the logs because I wanted to really demo. I had no clue coming back from maternity leave that there was this little not necessarily working or trying to say obstacle, a little obstacle to work with open source LLMs and the agent. So I was like, oh, I'm ready to go. I'm going to go ahead and demo this live. And I realized, oh my gosh, it doesn't work perfectly. So I was looking at the logs and it does have a system prompt in the beginning where I didn't use deep seek like you did. I used Quinn 2.5 and it'll say in the beginning, you're a helpful assistant, you have access to computer controller and developer extension, and you can do this, this, and this. I think another limitation is our hardware as well. So even though it's on a local device, and I mean it's supposed to work on a local device, our local devices might not have enough RAM or memory. I have a 64 gigabyte, but the person that came on the live stream with me, he had 128, so that worked much better. So that might've been a limitation for you as well. And even though the system prompt already told it what extensions it would have, we both had a better result when we started off the conversation saying, Hey, what tools do you have access to? And it probably referred to the system prompt and then went ahead and printed it out to us.Richard Moot: Yeah, yeah. And when I was tinkering with this, I actually was putting, I took an old gaming laptop basically set it up with, I converted it from Windows to Linux and it works reasonably well. It's still a little bit too slow for what I would actually want to be using it. So I have my regular gaming computer that I've actually, so when I want to mess with this, I actually just run a alarm on that when it's on, and then I use it as sort of a remote server and it's usable at that point. I think tokens don't fly through with the cloud LLMs. I mean it's still kind of slowish, but it's usable. And I think it's really fun to try out the local LLM stuff just, I mean as a developer, it gives me this mild peace of mind of my data's not going anywhere, so it feels safer somehow.Rizel Scarlett: Yeah, you're such a tinkerer. I love that.Richard Moot: Oh, I tinker with way too many things, network configurations, running clusters locally on my home lab, all stuff that I don't think I've ever used professionally, but I just love learning about this stuff.Rizel Scarlett: I love that. I love that.Richard Moot: So that kind of leads me one other thing that I'm interested in and I want to clarify. Not going to try to go into the realm of tips about using LLMs in augmenting our development workflows. And I think we're both in a similar camp of being in Devereux. It's really fun to just be like, I'm going to use this to start a new project or work in a language that I'm not usually familiar with and maybe see what I can build. I'm curious in terms of unprofessional tips, just things that you're sort of learning intuitively as you interact with it, how has it changed for you when you first started working with LLMs with doing software development to now? Have you ever noticed how you approach things a little bit differently?Rizel Scarlett: That's a really good question. I haven't thought about it. I know when I, let me think because when I started using, my first experience with LLMs was like GitHub copilot and I made this whole playbook for people to use. I was like, make sure you have detailed examples and stuff like that, but how has it changed now?Richard Moot: I'll give you an example. So to maybe help get your creative juices flowing on it. I know it's kind of coming out of left field, but when I first started using LLMs, I would just be like, Hey, can you build me this particular function? I think my first interaction was probably similar when I first used GitHub copilot and I was just doing tab completions and be like, I thought it was really cool that I could write a comment and describing the function that I want, and then I would start to write the function and then it would complete it out, and then I'd maybe have to edit a few things. And then once, I think it was when I first started using Goose is the first time I really tried one shotting things to be like build me an entire auth service for this app. And then I have now kind of swung back the other way a little bit where I tend to want to do a little bit more prompting when I want to do more of those one shots.Rizel Scarlett: Interesting.Richard Moot: But I've found that if I scaffold out half of it, maybe create the initial files and a single function or something and then it kind of fills in the rest, I have found it tends to work a little bit better. There's a few other tips that I can go into, but I don't know if that sort of helps. And that's how I've changed a little bit in how I've been using it because I've found that when you try doing the one shotting, it can just be too much that it's trying to do. And I feel like also it's too much context, especially I think one tip that I've heard continually with Goose and with others have a very clear what you considered a finished point and then move into a new context. Otherwise you can kind of go off the rails pretty fast.Rizel Scarlett: Yeah, okay. I'll say this. I think my experience might be a little bit different just because when copilot came out, I had to demo GitHub copilot. So I was already thinking of, okay, how do I do this one-shot prompt that'll build out this whole thing so that people can be super impressed with me? So I think I was doing a lot of one-shot prompts and I probably brought over some of that learning from there. ButRizel Scarlett: I think one thing I've been learning with building out the tutorials for Goose is kind of like what you're saying, how do I not let it get the context mixed up but still do a one-shot prompt? Because a lot of our tutorials, it's like we want to keep them short and sweet, so how do I make it do multiple things without overwhelming it or making it just fail? Sometimes it's like, I don't know what you want me to do, or it goes over the context limit. I leverage Goose hints a lot. So a lot of those different, most AI agents and AI tools have goose hints, cursor rules. I think Klein has its own thing as well. I dunno if you say Klein or Clean or whatever, but I dunno.Richard Moot: I don't know what it is either.Rizel Scarlett: Oh, okay. But they all have their own little, here's context for longer repeatable prompts. That way I use up less of the context window and I don't have to keep repeating myself like, Hey, make sure you set up this next JS thing. I'm already writing. We're going to use next JS and types script. We're going to use Tailwind or whatever it is, and then I can jump directly into the prompt that I want to do. And another thing I do, I don't know if this is weird, sometimes I ask Goose, how would you improve this prompt? I'll be like, I wrote this prompt and it failed. I'll open a new session. I'm like, how would you have improved this prompt? And it might give me a shorter one. So I don't know if I have these rules in my head, but I kind of just been really, really more experimental than I was in the past, I guess.Richard Moot: Yeah, I think that's a really good thing to call out. I think that that's something that I had learned over time where even though I try to figure out a way to codify an approach, but I end up realizing these are just, I mean, it's weird. I feel like I'm going to describe this, I am the LLM, but they're tools and then you figure out, yeah, I try this one and it's not quite doing what I want, so I'm going to go try something else. And then I think what you said there was one that I've even tried and I didn't really even think about is so go, sometimes I'll switch LLM providers and I'd be like,I'm trying to do this over here and it's not working, and I give it, it gives me something else as a different context. I'm like, well, let's try and see if I feed that back over here, if that gets me the result I'm wanting. And so yeah, I think that's the biggest importance right now is to just be continually experimenting with it because I think as time goes on, we're going to end up learning different, I don't know, heuristics of shortcuts of trying to get what we want done. Sometimes I really like doing the one shots and then other times I'm like, I maybe scaffold something out and then I'm like, I'm just going to progressively iteratively work through this because I don't want it to. I think when I was trying to have it build an off service for an app, I was just too worried about it in one shoting it that I'd have one or two tiny bugs that are hard to catch somewhere.Rizel Scarlett: That's true.Richard Moot: And then I was just like, I really don't want to have to be going back through all these different methods and figure out like, oh yeah, I'm actually handling the Jot token incorrectly here. I'd rather just sort of progressively work through and feel confident in it and then be like, okay, I'll give a concrete example of one where I was using this library, I think it was called, it's a one-time password library for node js. I didn't realize that it was kind of really outdated and not maintained, and I implemented it in this app that I was working on, and I realized it wasn't working in the way that I anticipated, and then I was like, oh, there's an updated version of this library somewhere else that's more well adopted and being maintained. And so I was writing out the conversion, but I only converted say one of the sign-in function, and then once I had that one converted, I basically told my LLM, Hey, I'm switching from this library to this library and I've already done this particular function. Can you go through and update all of the other ones to use this new library?Richard Moot: It did it, I would say nearly perfectly, which is pretty amazing. So I always find there's these little ways that you can be like, yeah, I'm going to do some of the manual work because it'll give me that confidence, but then lean on the LLM when I'm like, okay, I feel like I've done enough that it can finish it for me.Rizel Scarlett: You know what? Now that you say more, it does make me think, I think I use a different workflow if I'm building versus I am doing developer advocacy work for it demoing or doing a tutorial. So if I am building, you're right, I probably first ask it what is its plan? And I do go more iteratively and I do try to do more of it and then let Goose jump in at certain areas. But if I'm doing a demonstration, I want it to be I do a one shot, which is an art in itself for it to be one shot and for it to be repeatable because AI is non-deterministic. So it could have worked with me once and I tried to demo it and then it never works again, and people were like, this didn't work. But yeah, I think that iterative process is really helpful for me when I say like, Hey, how are you going to go about this? And Okay, I'm going to do this part. You do this. And I always open up a new session when I feel like the conversation's been going long because I think, well, I know it loses context as it gets too big.Richard Moot: Yeah, totally. I couldn't agree more on that part, and in fact, I've talked with some coworkers who've had mixed experiences with trying to use LLMs in their development work, but I think the thing that we just touched on there is that you have to just be dynamic in how you would use it and understand that you might not use it the same way in every context. And that's definitely what I've also learned when I've built out a fun example app for developer advocacy, just to build a proof of concept for the rest of my team to understand, hey, we can build an example and say next JS or Nest js or view or something, and I'm just using it to be like, Hey, I want you to one shot this out to basically get this mostly working to share something, but that's not how I build it When I'm like, Hey, I want to make this published and official for people to consume to say this is the way that you adopt Square. I would approach that very differently when using the LLM because I'd probably be curating my function signatures a little bit better and like, oh, this looks really good and understandable, and then have it fill out the restRichard Moot: Versus just one shotting things. If I was just going to have a one shot things, I would just probably tell people coming to our platform, so here's Goose, here's your LLM, go ahead and one shot your app.Rizel Scarlett: That's all you need, just Goose. There you go. And I think you had mentioned a little earlier that different, sometimes you'll switch to a different LLM in addition to experimenting with those prompts. Different LLMs have different outputs, and I know on the roadmap we're planning to come out with, I guess different products come out with benches of here's how well this LLM works with our tool, so we're coming out with Goose Bench to say, okay, maybe if you want to do this type of process or build this type of app, then maybe Anthropics models might be best or maybe opening eyes or whatever.Richard Moot: That would be really helpful because one other thing that I've been tinkering with when I try using an LLM for doing any kind of development work is I've actually, I think there's certain tools, I think one called Cursor New, but it basically just runs you through a series of give the project name a description, what libraries are using, and then it basically gives you prompts to feed in to create certain documents. But what I found interesting was the first one we'll do is say it'll help create a product requirements document, which I think we all call PRDs, but I'm just want to be super clear for people who might not know the lingo. And so I usually have it start out with creating the PRD, and then from there it'll create the code style guide and then it'll create your goose hints or cursor rules, and then I think finally it'll kind of create, I don't know how useful this one is, but it'll create a Read me of a progress tracker.Richard Moot: That one I've found it's cool, but I've not found it to be totally useful because usually the only time it is useful is when I finally get to say the end of a task where I'm like, Hey, build out, scaffold out the project, and then at the end of it I say, go ahead and go update the progress file to clarify what has been built, what should be built next, and where are we at. Then I use that as the start of the next session of, Hey, check the progress thing to see where we are and what we need to build next, and then work from there. That's so smart. I like that it's really been useful for me, but at the same time, I would say by this fourth or fifth session, I don't know why it starts getting a little, I run into too many errors and I don't know if it's actually specific to the number of sessions or the particular feature that I'm building is maybe too complex and I need to spend more time breaking it into smaller pieces.Rizel Scarlett: Interesting. I definitely, I want to try that on my own. I didn't think about saving it. I think the memory extension would do that as well for you maybe.Richard Moot: Yeah, to be clear, I think in this instance I was using something like Cursor.Rizel Scarlett: Okay.Richard Moot: But I think I was wanting to try this with Goose. I do have the memory extension enabled, but I just haven't actually gotten to this is sort of what I've been doing on my own at home, but I definitely want to try this more with Goose, especially because Goose has goose hints and I can very easily convert my other rules to work for Goose. But yeah, it's been very useful for larger, more complex things that I've been building, but I still feel like it has its limitations.Rizel Scarlett: Yeah, I like the challenge of figuring out what's stopping you and how do I get around this? And you're right, you did say you were using Cursor or some type of tool and I heard Goose in my head,Richard Moot: But yeah, I found it really helpful just trying to just experiment with all the various different tools. There's a lot that I think all of us don't know. There's a lot of stuff to just keep figuring out. I just think that it's weird to say to someone like, Hey, I could tell you the different ways that you could use this, but I think right now most people should actually just figure out how it can work for them because I think if you go online, you can find people on all ends of the spectrum. There's some people they work on stuff that's so bespoke complex that they go, oh, LMS are just not useful for me. They're too bug ridden or the performance of the functions it creates isn't useful. I think those people are one end of the spectrum versus other of us who are like, oh my gosh, I spend most of my day doing architecture stuff like API architecture stuff, and so having an LLM to do the actual implementation of a design is huge unlock because I might come up with a great API design, but then I'm like, this is going to take me so long to code up and an LLM feels like a huge unlock,Rizel Scarlett: And I really resonate with your point of figure out what works for you. You really just got to tinker with it and be like, I want to get this done and figure out how it'll apply to you because like you said, I could tell people one way, but I might not work in the same way as they do or I'm not working as complex stuff as them. Yeah,Richard Moot: And not to anthropomorphize it too much, but I feel like it's probably not too dissimilar if somebody just said, Hey, out of nowhere they said, Hey, we gave you an assistant. You'd be like, I don't even know what I'm, you'd at first be like, what do I use the assistant for? Can you organize my files? It takes you a while to even figure out, okay, what can you do? What are you good at? You don't know until you actually have that, and so I think we all have to just start interacting with it and then we'll figure out where we actually want help. You might find out there's certain areas where we don't want it to help us in these things, but we do want it to help us in those other things.Rizel Scarlett: Get to know the lm.Richard Moot: Exactly. Yeah. Even the people who think, oh, I don't really like it for coding. You're like, yeah, but you might find out you hate writing super long complicated emails or reading super long, complicated emails. You'd be like, Hey, can you go ahead and give me the TLDR of this or write an email form? It can be that simple.Rizel ScarlettSometimes I use LLMs to make sure my emails sound polite. Sometimes my emails don't come out polite, even though I'm not even throwing any shade, so if it sounds like I'm throwing shade, take it out.Richard Moot: That's great. I think more people should be using that. Maybe I'm biased because being in the Devereux space, you get to interact with all kinds of folks.Rizel Scarlett: Yes, that's true.Richard Moot: And they all have very different communication styles, not to name names. We did have one person who in this square community who I actually was very thankful for them, but they spent so much time just finding every single bug in our APIs, and it drove some people a little bit crazy like, oh my gosh, he found another one, but then I'm just sitting here just like, yeah, thank you. This is free qa. This is amazing. I'm going to keep encouraging this person to keep saying this stuff. I don't find it annoying in the leastRizel Scarlett: I could get it. I could understand it on both sides. As an engineer, you're like, no more work, but as a developer advocate, you're like, yay, my product's getting improved,Richard Moot: And it's validating. They love the product so much, they want it to be better, so of course, let's go do that.Rizel Scarlett: It's true. I love that.Richard Moot: Yeah. Well, I think we're coming up on our time here. Thank you so much for coming here and telling us a little bit about Goose. I think here's probably a good point for us to sort of plug where can people go to learn more about Goose Blocks Open source or if they want to just sort of follow you to learn more about any of this stuff.Rizel Scarlett: Yeah, I would, if I were an engineer or somebody interested in open source itself, I would go to github.com/block/goose, and if you wanted to go on the website or install it, I would go to block.github.io/goose and to find me on the internet, on any social media platform and Black gobys.Richard Moot: Perfect, and I will also just do the extra plug of check out the Block, open Source YouTube, and if you want, you can also check out the Square Developer YouTube if you're interested in things on Square Developer, and if you are working on a project or you want to learn more about what you can build or what is available on Square, go to developer dot square up.com or you can follow us at Square Dev on X. Thank you so much for being here, and we'll see you next time. 

  --------  

  40:21

More Technology podcasts

Trending Technology podcasts

About The Square Developer Podcast

The Square Developer Podcast: Podcasts in Family