Making FFXI An Offline Game.

At this point, it's starting to look like "can a fan group get together and make a FFXI clone".

I've looked into it a couple of times, and each time the 3D aspect of it kills it for me. I suck at math.

Key word... Not to be confused with "IF".

Honestly, I can't even imagine the crazy crap they'd have to pull just to make this game playable on a phone. I mean short of just attaching a controller or keyboard to your phone...

This or no ffxi, basically.

This game is fantastic, but sadly the A.I are programmed god awfully, and there is quite a lot of content inaccessible due to server closure sadly.

Last time I bothered to update the source and compile it, the "best" to offer of the private ffxi servers was not very good.

I don't understand if it's because there are just less dedicated or interested people working on it, for if it's because it's not been worked on as long or hard. Compared to the EQ Emu project, the FFXI one is not even walking yet. Hell, it's not even crawling.

Half of the people I see clamoring for a "classic" server have no conception at all of what that would entail. They just want someone else (Squeenix, generally) to do it for them. If all these people really wanted a classic server as badly as they say, they should learn C/C++/C#/whatevergodawfulvariantofbelovedcthatprojectuses and go work on it.

I mean, for Altana's sake, the only part of creating an FFXI clone that is difficult as I see it is doing the client, since I complete ***when it comes to doing the math to design a 3D Engine. Logic and internal mechanic design and various other server side and non-graphical client things are relatively easy. They majority of the hard part is just actually writing the damn code. Pretty much every technical problem has been solved, often multiple times, by different people, so you have a wide range of choices for solutions based on how much processing power and/or memory you want to spend, how many concurrent users, etc, etc, etc.

Hell, you shouldn't even be writing the code until your flowcharts are so detailed it's either "write the code" or "write the code on the flowchart".

I mean, "Login server listens on specific port for client to connect. When a connection is established, server sends message to client demanding client version, checking to see if it matches. If no match to current server version or other bad response, send error message and terminate. Otherwise, send message demanding credentials. Client transmits username and hashed password, these are checked against a database which contains the account data pertaining to login authentication and login authentication only. If the information does not match a record in good standing, send a response to the client indicating what the error is (bad username/password, banned account, subscription lapsed, etc). If information matches account in good standing, tell client to stand by. Login server sends message to lobby server requesting a unique ID from the lobby server that it can transmit to the client for identification purposes to hand the client off from the login server to the lobby server. Should such an ID be obtainable, it is sent to the login server, which forwards it to the client, along with the IP address and port information. The client disconnects from the login server, and attempts to connect to the lobby server on the specified address and port. Upon connection the lobby server demands the client transmit its unique temporary identifier. If the client supplies an identifier that matches one recently submitted that is still awaiting connection and has not yet expired, then the connection is accepted, blah blah blah blah blah blah blah blah blah blah blah."

That ***is not hard. And if you're rust with it (as I am), there's plenty of resources to refresh your memory. Honestly, the hardest part of server design would probably be load balancing. Either you need to dynamically allocate resources to zones as they require increased amounts, so if some linkshell decides to cram 300 people into a single tiny zone the zone doesn't crash, or you need to implement instancing of zones, or you need to implement a borderless world where sections are loaded in chunks as needed, which allows things to be split up, blah, blah, blah.

Again, none of this is *hard*. It's just work. All these problems have been solved time and time again with many different solutions. Some are better than others for specific purposes and situations.

Designing a 3D Engine, and then making models for it, and designing areas/regions/zones/whatever, and then doing animations: that is stuff I cannot do. I do not have enough artistically creative bone fragments in my body for that sort of thing. Making art assets isn't something I do, and writing a 3D engine from scratch isn't something I want to do, but then you either have to go with a free or nearly free one, or accept licensing fees, viral licenses, etc, etc.

Again, none of this is particularly hard (except, in my opinion, the graphical client parts). It just is a lot of work. Most of which would be flowcharting, and then documenting, and then writing ***tons of comments and tiny amounts of code.

What was the question?

So yeah, that was the question but that's still for private server gaming. What about a strict offline game that we could play at our leisure?

As for the other 3D models... well, they are already made! Just have to use the models and textures from the actual game. It's not like we're building from the ground up with nothing.

That is highly illegal.

It is?!

Oh. I see the difficulty rating just skyrocketed.

I've been following EQEmu on and off for more than ten years. I'm extremely overly aware of packet sniffing.

The reason EQEmu has been going on for so long so strong is because they literally incorporate nothing from the client. The only thing you even change in the client is merely a text file indicating where to look for the server.

The problem with dealing with private servers is that people with too little legal knowledge like to mouth off on them. It's perfectly legal to reverse engineer a piece of software. What's illegal is using appropriated assets, either in the form of graphics/music taken from the client, or source code.

If you have enough data to determine what all the communication between the client and the server is, and the "grammar" for that "language", then you can implement your own version of that server. It's just a lot of work.

And it's easier to *** on the OF that there's no reason why Squeenix shouldn't make a classic server despite being told those reasons over and over and over again because it's easier to *** than to do something about it.

You can copy the art style, you can copy the graphical interface's look, layout, etc. You cannot actually use any of the client's assets to make your new client for your clone, because you don't have the legal rights to use those assets.

If you could produce a functional clone of FFXI's client with almost exactly the same sounds, art style, interface, etc, etc, and could prove that you had created those things independently without using any of the data from the client, even if they are highly similar, you're fine. Unless there's some stupid trademark on mithra. Then you're boned. But you can't trademark catgirls. You could probably trademark the name "mithra", except that'd kind of be like trying to trademark "god". It's also a whole other can of legal worms.

That is why I think that the client is the hardest part of creating a client-server multiplayer game. What you are doing in creating a "server" is you're just defining the game logic, more or less. How movement is handled, how hits are calculated, what stats are, to speak broadly, down to things like "on hit effects are done via the effect registering a callback to the on hit effect handler for the player to whom the effect has been applied to and then the effects are sorted via priority and overwrite status" etc etc. Those are all relatively straightforward feats of logic.

Creating dozens of 3d models for the playable races and the monsters, and then thousands of different textures for the sections of those models for things like different armor appearance, sex, job, etc, etc, and then designing animations for them, not only a generalized animation for each action, but then specific variants of it for specific models and subclasses of the action (does a critical hit have a different animation than a non-crit? What about a mainhand crit versus an offhand crit? etc), that is a huuuuge amount of work that, while probably no less huge than writing the server, is not something I, or most other people, are good at doing.

For the past 4 years, I've been creating this MMO of mine and I can tell right off the bat it's going to be overwhelming when I get to coding the server aspect because of all the data that is involved.

I fully understand what you're saying and it's shed a spotlight on certain aspects that needs serious thinking.

It gets into licensing and intellectual property rights. Even if SquareEnix turns off the FFXI servers tomorrow, they still own the rights to the Final Fantasy trademark, the rights to the client source / binaries and the rights to whatever original server implementation (leaked source) was used. Using a program that bullsh!ts their client into pointing to a different private might not be considered fair usage. And someone using their client without first purchasing a valid license would definitely be considered illegal. Most of the time these homegrown old-MMO-emulation programs are just emulating the server mechanics and are small enough that the original owners either don't care or don't exist anymore. Earth and Beyond Emu is a good example of a very old MMO with a cult following being resurrected and maintained privately.

Flowcharts. Flowcharts are your friends.

Go buy a huge dry erase board. Get a couple of different colors of markers.

Then write on it, "GAME" in a big circle.

There's your game.

Then start adding things to it.

"GAME" would become "player logs in" which leads to "player plays game"

And then you just keep repeating that, over and over, breaking up the larger parts and concepts into smaller, more concrete fragments of what that involves.

Like all that crap I wrote above about the logic flow for logging in, that would all start as

"Player logs in"
"Player connects to Login Server, player logs in"
"Player connects to Login Server, transmits identification, logs in"
"Player connects to Login Server, transmits identification, credentials are checked, if good, player logs in"

It's all about what seem like big, monolith ideas and breaking them progressively down into smaller and smaller parts. Eventually you break them down to such a point that you have rendered them, more or less, into their basic "components" that are the things the code you'd be writing would be doing. And at that point, you make a record of all this documentation you've made, and write some code.

And you keep all of the documentation. You are bound to forget why you did certain things (why does that function take a void pointer?, why did I do this as an array, as a string, etc, etc), so you want to have the stuff you wrote down when you originally made that design decision. Ideally you'd also write up some notes for each flow chart section, too, and be sure to comment your code. There is nothing more aggravating than waking up hung over, checking the files, seeing there's been a new version compiled, that it runs, the problem you had was fixed, and then seeing the following:


That sort of ***is what makes you give up drinking.

Hmm coding server side stuff is far harder then that, depending on how scalable you want things to be. MMO's are giant database's with a front end application server. Client connects to front end server that converts the clients requests and database transactions, executes those transactions and returns the results to the client. For small connection pools this can all be done inside a single application, but once you start going bigger you rapidly need to develop custom flat databases (transactional is far too slow) and performance tuning gets really hard. It's very easy to engineer oneself into a corner where you can't scale past a certain point without astronomical system requirements. A game like FFXI has some pretty ridiculous back end requirements, tens of thousands of transactions per second for a moderately sized world, more for a busy one. It's so much that you have several dedicated servers per world with each one running different sets of zones and each zone having a set resource limit. Building a simulator for that is going to be a nightmare without access to the original engineering documentation or source code.

No, you don't do tens of thousands of transactions a second to the database. You keep snapshots, and do occasional massive transactions instead of constant massive transactions.

You don't need to be doing a second-by-second backup of every thing in the game state to the database.

Umm ... I think we're using different definitions of database. I'm not talking a relational SQL database but a collection of massive data structures that represent the active state of everything in the game.

Everything in memory is a database in an MMO. Every position of every object, every unique instance of an item, every unique instance of a monster, every action executed, every script that's run, it's all stored inside an active database. In a single player or even small multiplayer (ten or less) game you don't need to do this because there isn't much synchronization that needs to happen. Once you surpass that you quickly run into a problem where too many actions are being executed simultaneously for single logic loop threading to be remotely possible. You need the logic for every action to be run separately and simultaneously, this cause's the need for a massive central set of data-structures that are used.

Essentially you attacking a monster, your friend casting a spell, and the monster's AI are three separate logic threads running independently of each other which need to reference a single global datastructure to maintain consistency. This is done in a transactional fashion to prevent one action from overwriting the data another action is reading (you moved when the monster executed an attack). The server this is running on needs to be capable of processing tens of thousands of those transactions per second to maintain consistency, otherwise you experience action "lag", where one thing you do isn't updated for several seconds causing other elements to operate under wrong assumptions and everything gets out of sync.

Every time you move, there is a transaction. Every action you submit there is a transaction, every action the monster submits there is a transaction, every item that drops is a transaction lookup, every lot, every inventory action, practically everything done server side will be a transaction to this global database. With a thousand players logged in simultaneously, how many transactions are being conducted every second, every half a second, every 100ms?

Or do you seriously think everything is being run on a single main{} logic loop that checks everyone's actions and the results thereof in a serial fashion?

Okay, first of all, I don't think we're on the same page here.

The server that is responsible for me running around in Valkurm Dunes is not the server that is responsible for you spamming cure on yourself in Port Jeuno.

The amount of action that actually takes place is extremely small. You are seriously underestimating how many people are actually being serviced by a single 'server'.

The server we play on, Asura, is composed of multiple separate server processes. One for each zone, one for the auction house, one for global communication, there's the instancing. There is not a single "Asura server" -- there's dozens.

While there maybe possibly be several thousand "transactions" going on per second when you have a couple thousand players connected, these are not all being handled by a single process. They are split up in a logical fashion between multiple machines running multiple different processes.

The zoneserver for Dunes doesn't need to know jack ***about what is going on in sea. All it needs is the data for the players in Dunes. Even if you have 300 people in a single "area", they're not going to be generating tens of thousands of actions per second. At best you're looking at each person generating, maybe, 10 actions per second if they're just slapping their keyboards.

There is no need whatsoever for a Bastok Markets zoneserver to give a ***about chat going on in Konschtat Highlands. /say and /shout are area specific. /p and /tell would be sent from the zoneserver to a global chat server which would ensure that each player got sent the appropriate messages.

If you go sometime and take a look at the EverQuest Emulator project's server, and how it splits everything up into multiple, separate processes that don't need to be on the same local machine but can be split up across different machines in different physical locations, that's how you do it. It's all about compartmentalizing the information.

A "single main{} loop" is more than capable of accepting requests from 200 to 300 concurrent users and processing that in a FIFO fashion. You split the game world up into several dozen different zones, possibly instancing the more busy ones, like towns, and then split these zoneservers up across multiple machines. When a player is transfered from one zone to another by the world server, the zoneserver retrieves their player information. Zone servers don't need to know that any players exist who are not in their zones, nor does the global chat server need to know any players exist who aren't logged in, etc, etc.

The problem seems to be that you are coming at this from a monolithic standpoint, whereas I am looking at it as dividing things up to spread the "server" over multiple processes and machines.

Umm I'm approaching it from precisely the requirement to have everything split up. I'm telling you exactly how you need to build a game of FFXI's scale and the server back-end is most definitely not the way you describe. It wouldn't scale very high until you hit memory access contention.

I want you to take a look outside bastok, count the number of monsters moving around, notice that they move regardless if someone is nearby or even present in that zone. That is an AI thread running that is constantly checking it's state against a global dataset, it's also moving itself around. An aggressive monster AI thread would be checking the position map of the zone and seeing if any player object was nearby. How many monsters are in a zone, how many AI threads would be running simultaneously with the player threads? A single battle would produce quite a bit of concurrent memory access, how many battles could happen in any particular zone? How would you account for the potential results of all this and how to keep it synched and consistent?

Your approaching this as a handful of people running around doing random stuff, I'm approaching it as a system designed for a few thousand simultaneous players doing independent actions and it not crashing. Scaling is a motherf*cker.

As for the servers, that is not how that works at all. What you have is a handful of servers (4~12) per "world", these are usually large machines but four to eight core single socket solutions with shared storage lets them rather small, with a very wide performance capability (lots of independently addressable cores running at low speed). Inside these machines are a bunch of instances of a zone server, this is similiar to how virtualization works but a form of environmental virtualization vs machine virtualization. Each of those instances can be shut down independently of the rest, they can be migrated during maintenance windows for better system resource utilization. Each of those zones maintains it's own global database that is also synchronized with a global master database for that world. This is incredibly important for things like inventory management and server wide communication tracking. Every action in those zones is a transaction against the zone database.


Dumbest thing possible in a large scale MMO. That is precisely how you crash a zone and introduce all sorts of bugs and glitch's into the game. That also fails miserably when scaled up to production size implementations. In a large scale system design the last thing you want is a "deep" implementation, which is a single heavy thread running everything and thus a single point of system resource contention. You want "wide" implementations, which are many concurrent threads operating simultaneously and thus spreading out the I/O requirements. This is a whole different class I could teach. What works best for smallish systems doesn't scale well into a larger system and requires a different type of engineering.

I was specifically referencing your comment that server side work on an MMO is easier then client, which is quite wrong. Client side work is almost entirely dominated by professional graphics artists, which are not software engineers who themselves are not systems engineers. A successful product of this scale doesn't get made by a single person sitting in their room, it requires an entire team of specialists to engineer and build it. The client is nothing but a terminal that express's what the server tells it while telling the server what the user wants to do, it's very easy to engineer. The server side of the system needs to take the requests from all the different clients and determine what to do with them, while also maintain stability and coherency. Those are extremely difficult challenges as you scale up users. The words "sorry boss, we are having issues with that new product because I didn't design a robust enough system" is a very good way to get fired and the company to lose a few million USD.

Now if we're only talking about an enthusiast implementation then it's not much of a problem.

And by the way, system =/= a single server. That's one of the easiest ways to spot an amateur. A system is the entire thing taken together. Systems need to be engineered with scalability and robustness in mind because once implemented they are notoriously hard to re-engineer without causing massive downtime and costs. It's often cheaper to just start from scratch. FFXI's reliance on PS2 devkits and "PS2 limits" are examples of bad systems engineering.

So why we discuss this when producer already said there is no offline game planned?

I'm an absolute HUGE fan of the White Knight Chronicles franchise (Platinum for I and II) and was online right up to the day they shut the servers down. I agree that there are similarities to FFXI and offline dynamics that would work if FFXI followed suit - at least in some capacity.

Sure, the AI in WKC wasn't perfect, but the game was fun and worthwhile. If Level 5 ever confirms WKC III I have to say.. I may take a break from XI.

If SE isnt going to care about what happens to ffxi after it shuts down (which we dont know yet how much longer it will take), private server people are going to do it and that is probably even less in the interest of SE. mobile game pfff what am I gonna do, gearswap on my smartphone via touchscreen? yeah right.

I believe the ilvls totally killed ffxi since it just doesnt go with how ffxi worked. keep the ilvls to ffxiv where they belong and where they work. Therefore, Id find it naturally to cater peoples different wishes on "which" ffxi they want to play:

one solution I can think of would be SE would start to offer ffxi servers at different stages of the game. Like one/few servers at the 75 era, another one/few servers at 99 without ilvls and the rest of the servers at the current level with ilvls.
(if you like to transfer say ilvl server to 75 server, you get capped and your equip 75+ rendered unusable). Theres a ton of details to think about like keep certain patches like telepoint stuff still for 75 server or records of eminence even?

one step further would be SE allows people to host servers, maybe for some extra money, and allow the to adjust the game to what they believe is the best ffxi. Then you would be able to pick between numerous servers, none like the other, which comes closest to your idea of what ffxi should be like.
(And no I didnt make a precise plan on how that should work, be funded, appraised by SE or all these things lol Im just having ideas and Im sure there could be a way to implement that)

even one step further would of course be private servers. SE either selling the game like an offline game to people that may open servers then, for a small suscription SE could even make money by providing the platform where all these private servers are coordinated.
The problem I always see in private servers is you play maybe 2 years, build up your char (thats oldschool ffxi terminology, new players might not know what "building up" means :P ) and then the host thinks about doing something else or might even die and the server is gone and your char with it.
By SE providing a platform that so to say supervises these private servers, they could guarantee you to save your character data even if your server shuts down and allow you to switch to another server you might find similar to the one you played. that would be (at least for me) a reason to pay them a monthly fee as opposed to free private pirate servers. also: if its under SE supervision, its original code so stuff works.
This way, SE would have to do next to nothing, maintain a platform that people can log on to and get forwarded to their servers and maybe keep track of the chars accomplishments on these servers which should take more HD space than a textfile each. And they would still gain monthly fees. not as much as ffxi used to but more than nothing which they will when the game does shut down or people just go to pirate servers.
I guess getting money for such little work should beat getting nothing for doing nothing for SE. Blizzard does it even for free concerning diablo II,III starcraft warcraft stuff (except WoW of course).

The reason none of the XI private servers are on the level of other games is because people don't understand XI was just a blip on the MMO scene in the grand scheme of things. Games like WoW/EQ/UO/SWG and other games with emulation servers honestly had playerbases that far exceed our own in numbers. This means these other games get to enjoy lots of players with technical knowledge and skills that are invested in these games, and that means they're more capable of reliving old memories, as such a thing would be a challenging weekend hobby for these types of people.

Meanwhile in Vana'diel, all the really technical players seemed to have become bored with the game ages ago and vamoosed to other things in life, leaving behind a shrinking group of players clamoring for "the good ol days", hoping SE or someone else will make the impossible possible. The end result is that even if an offline XI somehow happens, it really won't fill the void. At best, it will merely be like giving a junkie another needle, at worst, it will forever serve as a painful reminder of what once was. The era of NASA scientist making bots for fun and sports car money from a hobby of XI are over. At best, we got Windower people that have an inkling of how things work within the game, but they're really just arm chair theorist compared to the kind of people that others are really hoping for in this thread.

Keep in mind that SE is also excessively greed driven, and has ruthlessly gone after fan projects in the past, and will do so if any kind of offline isn't created and sold by the house Sakaguchi helped build (and nearly killed) with Final Fantasy. I'd like to use a more personal example, where a small group of die hard FF fans actually took the time and created a working FF4 MMO.



Functional ATB, each member is their own person with their own ATB and commands, and all the enemies were painstakingly recreated with proper drop % and tables as they were in the real game. Even had added stuff like Blue Mage / Red Mage, and other things to make it more enjoyable and FF homage. This was all done in BYOND, so recreated from scratch, and a true labor of love. What does SE do? Shoots a C&D for a game that honestly had maybe 30 players max, and causes the game and its close knit community to disperse forever. Some could argue that it was competition to XI, due to being a free alternative, but honestly it was just a *** move to be a *** move.

Hope and pray SE is the one with the offline XI, since there is no future otherwise. They guarantee it.

They should absolutely make XI an offline game. yet still make going online with people a very viable option.

Weren't people talking FFXI ending like 10 years ago? Even if SE doesn't make any more expansions to FFXI, people will still play. Square-Enix just needs to advertise FFXI more. Long live FFXI!

As for an offline FFXI... it should be a side-story or 8-bit style graphics like this: http://www.ffxiah.com/forum/topic/36362/8-bit-ffxi/8/ <- SE could give some devoted fans several thousand dollars to finish / work on something like this. ^^;

Meow!
-Skjalf, Daughter of Eir :3

I don't know why I even bother replying to someone like you...but people will drop out fast once the content stops. How long it'll last depends on the last grind implemented in november, but anyone with a brain can see how the game will plummet quickly.