CubeNet M2: The ResNet Revolution and Optimization

Abstract Following the experimental success of CubeNet M1, CubeNet M2 represents a paradigm shift from "Brute Force" to "Architectural Efficiency." While M1 relied on the raw power of an H100 GPU, M2 is designed to run efficiently on a single consumer hardware (RTX 5080) by leveraging ResNet (Residual Networks) and Mixed Precision Training. This research focuses on overcoming the stability issues of M1 not by adding complex RL tricks, but by refining the neural backbone and the training curriculum.

Introduction

CubeNet M1 has proven that a neural network could solve a Rubik's Cube, but it suffered from Catastrophic Forgetting and inefficiency. The "Dense Escalation" CNN used in M1 struggled to propagate gradients through deep layers, and the training process was computationally expensive.

CubeNet M2 was built to solve these structural flaws. By implementing a ResNet, we enabled the model to learn deeper, more abstract geometric relationships without the gradient vanishing problem. Furthermore, by optimizing the training pipeline with BFloat16 Mixed Precision and fully vectorized environments, M2 achieves higher throughput on a 16GB VRAM GPU than M1 did on an 80GB H100.

CubeNet M2 benchmark comparing training throughput and solve performance with the previous model

Architecture & Optimization

1. Neural Network: From CNN to ResNet

One of the most significant upgrade in M2 is the "Brain".

M1(CNN): It treated the cube state as a simple pattern matching problem.
M2(ResNet): By adding "Skip Connections" (Residual Blocks), the network can preserve the identity of the input features through deeper layers. This allows M2 to understand complex, multi-stage dependencies that the M1 architecture simply couldn't grasp.

2. Algorithm: Optimized & Updated Standard DQN

Unlike the theoretical "Rainbow" agents, M2 sticks to the robust Standard DQN algorithm but optimizes the hell out of it. We wanted to leverage the performance with it's potential, not just by implementing a great pre-made algorithm.

Review Diet: M1 failed because it "forgot" how to solve Level 1 cubes while trying to solve Level 14. M2 introduces a Curriculum Learning update with a Review Diet. The training loop proactively mixes in easier scramble levels during advanced training. This ensures the foundational neural pathways remain active and robust, preventing the "skill collapse" seen in the previous generation. This technique helped the training mostly, even outdoes the neuron network upgrade.

Review Diet curriculum mixing easier scramble levels into later training stages

Epsilon Trap: A novel exploration strategy where epsilon (randomness) is adjusted specifically at the first step of an episode. This allows the agent to stack good initial moves (exploitation) while still exploring new branches later in the solve.

Epsilon Trap schedule reducing randomness on the first move while preserving later exploration

Action Masking (Oscillation Resolve): We solved the "Infinite Loop" bug where the agent would toggle moves (e.g., U -> U' -> U) to farm rewards or stall. M2 explicitly masks the inverse of the previous move, forcing the agent to commit to a path.

Action masking blocking the inverse of the previous move to prevent oscillation

3. The Hardware Engine: VRAM Optimization

M2 is an engineering marvel. It moves the entire Replay Buffer into VRAM, eliminating the slow CPU-to-GPU data transfer bottleneck.

Mixed Precision (BFloat16): We cut memory usage in half and doubled computation speed without losing accuracy.
Throughput: This allows us to simulate thousands of cubes in parallel on a single card, stabilizing the sparse reward signal through sheer volume of data.

Conclusion

CubeNet M2 proves that we didn't need the most complex algorithm (Double/Dueling/Rainbow) to get results if the Backbone (ResNet) and Data Pipeline are engineered correctly. It is a leaner, faster, and smarter successor to M1.

However, M2 is still a probabilistic solver. It plays "by feel". To achieve perfection (100% solve rate), we must transcend the limitation; The instinct problem solving.