Request for comments on a new major API version proposal

Taking STiBaRC back to its roots, it's time for some meta-discussion about the platform on itself. I've been working on a new major version of the STiBaRC API (v5), and this time I've started with the specifications for it before the implementation. I'm trying to make sure I capture all the existing functionality of API v4, while providing an actually consistent (and documented!!!) interface. That being, a post, comment, or user looks the same everywhere it's used, both in responses and requests. API v4 follows the traditional model of STiBaRC API design, which is a very PHP-like approach of making different SJS files for each type of operation there is to do (getpost.sjs vs newpost.sjs, for example). The API v5 proposal takes the approach of REST-ful API design by providing endpoints for each resource (user, post, clip, comments, etc.), organizing related resources under sub-paths, and using the HTTP method to change what you do with that resource. For example, POST /v4/newpost.sjs would become POST /v5/posts, and GET /v4/getpost.sjs?id=4102 would become GET /v5/posts/4102 This also moves the session token out of the body and into the Authorization header, where it should have been the entire time, so requests that only wanted to get information no longer have to be a POST request instead. It also eliminates the useless (because we still used HTTP status codes anyway) "status" property of the response in favor of providing a proper error response with 4xx and 5xx series status codes that follow RFC 9457 "Problem Details for HTTP APIs". This gives developers a reliable and descriptive type to check against (which links to API documentation about the error), and user-facing information the application can show without needing to map error types to their own descriptions, under the "title" and "detail" properties. Other large changes: - Splitting off a user's posts, comments, clips, followers/following into separate endpoints. - Splitting off post comments to its own endpoint - Pagination on getting lists of resources (like search, global posts, user posts). - Per-resource scopes and read-only versions of them. Existing scopes on sessions will be migrated to the appropriate new scopes. Splitting these off into their own endpoints does not come with any major performance penalties due to request pipelining and parallelization, and allows for pagination without repeating common data (like profile or post information). Posts and profiles should load even faster actually, since common information will be able to load and render first without waiting on the comments/posts. And this allows API users to get only the information they care about. The plan is to continue supporting API v4 for six months post v5 completion, before being removed. This should be plenty of time to move off, but I can extend the deadline if necessary. I tried to make it convenient to migrate, since the request and response bodies should be largely compatible with v4. But before I even start with any of this, I want your feedback! You can view the API details at: STiBaRC: https://editor.swagger.io/?url=https%3A%2F%2Ff-up.joshiepoo.com%2Fstibarc_api_v5%2Fapi_v5_oas.yaml Identity service: https://editor.swagger.io/?url=https%3A%2F%2Ff-up.joshiepoo.com%2Fstibarc_api_v5%2Fidentity_oas.yaml Pre-gaming some questions: Q: I see some features on the new API that don't exist now. Do we have to wait for the new API to use those features? A: No! The datastore API and notifications are planned for API v4 right now, I just wanted to include them in the API v5 proposal so they aren't forgotten about and have a clean implementation. Q: Why is there a separate identity service? A: To finally unify www.stibarc.com and stibarc.com, keep you signed in across other future services (like Messenger), and make it easier to manage multiple accounts. The idea is that your identity.stibarc.com account is one thing, and your existing STiBaRC (the social media service) accounts are connected to it. The migration plan would involve automatically creating an identity account for each existing STiBaRC account, and then users can choose to combine them together if they want to, picking which one they want to be the primary account. From then on you only have one password to worry about. You'll have the ability to transfer them between Identity accounts if you'd like, which also lets you effectively disconnect them by making a new Identity account with nothing but the one STiBaRC account. Also I wanted to move developer apps to the identity service so they can access Thinkblog resources as well. Q: If I still want account switching without linking accounts, will that still be possible? A: Yes, identity.stibarc.com will allow you to be logged into multiple independent Identity accounts at the same time. Q: I thought Thinkblog was going to have its own, separate account system? A: It still is! As promised, Thinkblog will continue to have its own account system, and will not require an identity.stibarc.com account to use. But the option to link it will be there if you want to, and it will allow you to use the same credentials across both services and manage them together. Using OAuth apps created in the Identity service for Thinkblog will not require end-users to have an Identity account, but to create an OAuth app that can access Thinkblog resources, you will need to have an Identity account. This is so I don't have to create a separate developer portal for Thinkblog. Q: Are you still working on the Thinkblog redesign? A: Yes. I'll probably try to finish it first actually. Identity account linking will come after that. Q: When are RSS feeds coming back? A: Sometime.

@sophie burden of maintenance, new features may become available in v5 in the future that won't have any parity for v4. Hence why I want to make sure v5 does everything v4 already does.

What is the reason for shutting down the v4 API after 6 months?

@Eiim I appreciate the constructive feedback! - Haven't determined exactly what values Notification.type will have yet, but the intention is to make it an enum once I know. - Error.type is indeed a unique (per error type) URL that points to API documentation, which does not exist yet! Because there's a generic Error type for the entire OAS (and because I haven't yet made the entire list of possible error types as there could be any number of validation problems), I've opted to not list all of those in the document. I will probably at least enum the type once I have that list, but not the rest of those values (you should really only care about the type and pipe forward the other values anyway). Also, this is not the only documentation that will exist, I really would like to have HTML documentation as well. OAS documents are inflexible in that it makes it difficult to easily tell you exactly WHICH error types a call can produce, so that information will be relegated to the HTML docs. This OAS is more a schema to tell you what data structure you should send and expect back. - All IDs are strings instead of numbers because I cannot promise you they will always be numbers and want to prepare you for that. Post IDs will probably stay numbers, but comments? They don't have to be. They might become UUIDs. Best practice is don't treat them like numbers and sort on something else, like the date field. - Comments are still on the post/clip types because I forgot to change them to be the number of comments on the post/clip after making the /comments endpoint. Looking at the data types, Post/Clip and SmallPost/SmallClip are practically identical with the removal of comments, so I'm just going to remove the Small* type. Unless you can think of any properties to drop from a Post/Clip. - Good point on Attachments, changed to Attachment instead. If there's any metadata added later this type can change.

Looks cool! Notification.type should probably be an enum. If it needs to be open-ended, common values should at least be documented. It looks like the Error.type value is a URL pointing to an API request, but that part of the API isn't documented. Is there a reason why IDs are strings instead of numbers across the API? Why does the Post object have a list of comments if you're "splitting off post comments to its own endpoint"? I think there should just be one Post object which lacks comments and you have to get /posts/{postId}/comments if you're going this route. Same for Clip/SmallClip. If you do have the Post/SmallPost split, let us request a single SmallPost where we want it for applications that want that. I've seen this implemented like /posts/{postId}/simple. Also, if you keep this split, why does SmallClip not have a comment count like SmallPost? Also, I think that the comment count field should have a different name to avoid confusion with the comments field in Post. Attachments feels like an odd type, it would make more semantic sense to have Attachment as a wrapper for string and just use an array of Attachment. Although the resulting JSON is more sensible the way you have it. Not sure.

Question I forgot to pre-game: Q: Why make a new major version at all? A: V4 is hard to document and internally inconsistent all over the place (lots of endpoints working on the same resource expect different inputs). Many endpoints also handle multiple operations on the same endpoint depending on the request body, which is bad practice, and makes it even harder to document. V5 breaks these into their own endpoints so it's 100% clear which endpoint you need to use to do one specific action. This would not be starting from scratch, however, I've tried to keep the data model close to what's already being used, and I'm able to re-use a lot of the logic already used in V4 to implement this.

Not all of these API endpoints are meant for third party developers (especially Identity service ones, as there shouldn't really be third party clients for that), but they're documented anyway for my purposes, and to outline the concepts of the proposed new system

Request for comments on a new major API version proposal

6 Comments