Online Game Trade Engine

Building a cloud-based, scalable event-driven analytics pipeline to understand player trading behavior in a hypothetical online game

Description

The initial objective of this project was to simply build a lightweight analytics pipeline that captures specific events (purchases of items) in a hypothetical online game. That was a bit too easy, so I extended the idea to build a pipeline for tracking trades between characters in the hypothetical online game. Turns out it was necessary to develop the whole trade engine in order to capture the events I was interested in at the level of detail that would be applicable in the real world. The components of the engine that were built include:

  • character creation
  • trade validation
  • trade execution

In addition to enabling the trade functionality itself, I wanted to be able to track: - trades between players - items and currency included in each trade

Techniques

  • APIs
  • event-driven pipelines
  • message queues
  • caching
  • streaming

Tools

  • Flask
  • Redis
  • Kafka
  • Spark
  • Hive
  • Docker
  • Google Cloud Platform

Outcome

Built a Flask API to handle character creation and trades between characters. Enabled fast trade validation by caching player state in Redis. Pushed trade events to a Kafka topic, which were transformed using Spark Streaming and stored in HDFS. Analyzed behavior by querying with SQL using Hive.

More Information

More information can be found at the following links:

GitHub Respository: https://github.com/nsylva/understanding-user-behavior