After years of writing log lines and trying to find a perfect format, I've come to a conclusion: Always log JSON.
Logging to JSON makes your code read more consistently, you save time thinking about how to log your message and it allow you to separate concerns between logging data from processing it for various use cases like debugging, generating reports and performance.
The relationships between Collection Points, Collectibles, Blades, Field Skills, Regions, etc. are all linked up so you can easily navigate between them and find related pieces of information.
It's probably not very noticeable on a desktop computer, but I've optimised a lot of the rendering logic throughout the app. There were a lot of unnecessary calculations going on when navigating around, causing a slightly longer wait time. As more data is imported, it gets more noticeable.
Things should feel a lot smoother now. There's more improvements to come in the same vein.
I've started importing Enemy locations to the Xenoblade 2 Maps. Before I push that out though, I'll need to introduce a Legend toggle of some sort to reduce the types of markers on screen.
It seems pretty straight forward at first, but there's a lot of decisions to make. Things like:
What markers to show by default.
What to show when a user comes from an external link.
Whether to persist visible markers so that link sharing is consistent between users.
The most convenient place to put the toggle menu for desktop and mobile interfaces.
As with everything, I'll decide on a few things and improve it as I go.
April saw a steady progress towards open sourcing FrontierNav and the gradual introduction of collaborative tooling. The interface has been tidied up too: consistent colours, cleaner layout, better navigation. On top of that, Xenoblade 2's map is almost ready!
It's nearing the end of May now but it's worth mentioning May's update will come in June as usual with Progress Reports. This update was delayed by some weeks.
Xenoblade 2 Maps
The data for the maps is available now. It's just a matter of linking it all up, making sense of it and making it presentable. Can't wait!
With the recent rush for GDPR compliance, services are becoming more aware of the amount of data they hold and if it's really necessary to have all of it.
Application logs contain a variety of historical data coming from both users and third-parties, making it extremely useful when running reports and to monitor production behaviour. However, after a certain period, the burden of responsibility will begin to outweigh the usefulness of the data. Once that point is reached, it's best to shed that responsibility.
A common way to store logs is to put them on AWS S3. But, without the proper configuration, those logs will remain there indefinitely. You could manually delete objects or set an expiry when they're uploaded but there's an even more convenient solution built into S3: Lifecycle Rules.
Apparently, the current implementation may cause Firefox 60 to crash. This has been fixed in Firefox 61 which will release in June 2018. So if you encounter any crashes, set network.trr.mode back to 0 (i.e. off).
Recently, I've been thinking about browser support for FrontierNav. Most browsers support native ECMAScript 2015 (ES6) code natively, but somewhat popular yet unmaintained browsers like Android 4.4, Internet Explorer 11 and Chrome 29 have been holding things back.
So, I decided to drop support for them and go full ES6 in production. I thought the ecosystem and build tools were ready. I was wrong.
Following on from last month's update, this month has been a stable continuation in all aspects. More code has been split out towards an open source future, more of the data pipeline has been fleshed out, and more visualisations are being introduced.
The (now on-hold) Wiki platform did a lot of the initial work for the new maps. And with the recent progress in datamining Xenoblade 2, having all relevant locations in place may not take too long.
What kind of things are possible? I'll be able to show some of those once there's more data to work with.
I've looked into ways to get more community engagement on FrontierNav. One thing I mentioned before was the ability to create and import text-based guides. It's simple and a pretty useful way to dump information. So I'll be implementing this using the work I already did on the Wiki. It'll be a gradual roll out to reduce any moderation burdens it may lay on me.
So turns out all the Wiki work wasn't a huge waste of time, it just had to be broken down into smaller features!
When I first built FrontierNav, it was only a map for Xenoblade X. I didn't plan to make it anything bigger, so a lot of it was very ad-hoc. But over the months and years it's been growing slowly into something a lot more dynamic.
A lot of the conversations I've had about FrontierNav can get a bit difficult. Right now when people land on FrontierNav, it's not very obvious what it provides. And that's expected, since it provides what I felt like making at some point in time. But at the same time, I have a long-term goal for FrontierNav which I fail to describe succinctly.
So, here's the shortest description I can think of:
FrontierNav is a data management and visualisation platform for video games.
Is that too generic? Maybe. Is it realistic? I mean, sure the tooling isn't there yet, but it's getting there. Each new piece of functionality is a step towards that goal.
This month I'll implement the Text Guides interface hoping to capture the Pastebin audience and get to work on the Spreadsheet interface to capture the Google Sheets audience.
Once Xenoblade 2's map data is available, I'll focus on getting the maps ready for players to use. In addition, I'll link it up with the Affinity Charts to make completion a lot more streamlined. So for example, "Kill 5 enemies" clicks through to a map of where those enemies are.
At some point, likely not this month, I want to try out releasing a Dat version of the web app. Though experimental and not really popular, it seems ideal for FrontierNav and it would be a neat proof of concept.
Leaflet is an excellent library for creating interactive maps. It's one of the core technologies I'm using in FrontierNav, and one of the most mature browser-based mapping libraries out there. FrontierNav eventually moved over to React, so ideally I'd want a mapping library that also used React. Sadly nothing's really as mature as Leaflet, so I stuck with it.
React and Leaflet Interoperability
One of the issues I came across after moving the project over, was trying to integrate React's update logic with Leaflet's custom markers, dubbed "DivIcons". For a good year, I left the logic intact, using custom logic to manually manipulate DOM elements by subscribing to a Redux store. It worked but it was complicated and a pain to revisit.
Eventually I moved over to react-leaflet, a library which wraps Leaflet and provides pre-made React Components to represent various Leaflet components (Maps, Tile Layers, Markers, etc.). This was a huge improvement. While it was still using Leaflet's API under the hood, the update logic and state diffing could be handled in React like every other component in the app. Perfect!
Except, there was one major issue. react-leaflet does not support React-powered DivIcons and it doesn't plan to. The aim of react-leaflet is to be a thin layer on top of Leaflet's API without any additional features. So, what can I do?
Previously, FrontierNav only had one visualisation: a map. With the new Affinity Charts for Xenoblade 2, I've been looking into various approaches to present the same information along side different visualisations.
For example, showing both the Affinity Charts, and allowing users to navigate to the map while retaining their view of the "entity" they're looking like, like the Affinity Reward or the Enemy requirement.
There are a few candidate ideas I'll be fleshing out a bit.
One's a windowing system like any Operating System with the visualisation as a background.
Another is a "bubble" system, kind of like Facebook's Android app and Samsung's Touchwiz window system.
More raw data has been coming out for Xenoblade 2, so I've started writing a parser to transform it into a compatible format for FrontierNav.
Keeping track of all of this work and keeping it organised has been a bit of a pain. I'll be organising frontiernav-data to be a bit more, well, organised.
Due to the number of moving parts and complexity of the project, open sourcing it as-is isn't as worthwhile. With the lack of documentation it'll be difficult for anyone to contribute or make use of it.
So, to get the web client into a state that can be open sourced, I've been splitting out bits into their own project. A mono-repo approach has worked well for this as it means I don't need to juggle multiple Git repos and branching dependencies.
I'll be continue to work on the things outlined above. There's a lot to do but I'm getting a better idea of where to take FrontierNav by doing it.
Having to play new games, enjoy the first playthrough, then figure out their intricacies while also extracting data and building new features in a timely manner is really difficult.
Once Xenoblade 2's data is imported and guides are stable, I'm thinking of moving onto more classic games. The main reason being that all the data is already available and I can use them as an opportunity to build FrontierNav into a state where it's a lot easier to add new, recent games.
What games? There's quite a few. I've had an itch to replay Golden Sun Books 1 & 2 recently.