#3:
When I first played Star Fox 64, I didn't "get it". The game seemed ok... but nothing special. It was quickly ignored in favor of other games. Years later I played again and was amazed at the varied mission structure, the branching paths, the secrets in each level, and the overall polish. How was I able to just write that game off so easily? Now it stands as one of my favorite N64 games of all time.
#2:
Everyone talks about Mario 64 being a peerless classic. Totally disagree. I owned Banjo Kazooie and played that game before trying Mario 64, and still feel they took everything good about Mario 64 and improved on it. The graphics and sound design were better. Each stage was huge, filled with secrets and challenges, and had hundreds of items to collect. Even the controls were tighter, even with all of the special moves available in Banjo Kazooie. I understand that Mario 64 came first and set the bar high... but Banjo surpassed that game on every level.
#1:
Mario Party. Not as technically impressive as other popular games. Not a series that is really considered a "classic" by most. But it started the trend of mini-games, spawned many sequels (and unofficial rip-offs), and was the center of attention for endless afternoons with me and my friends. This game crossed age and gender boundaries better than any other game since Tetris to bring everyone together for some laughs and friendly competition. That is why it's my pick for the best game on the N64, despite broken analog sticks and blistered palms.
Games that I'm sad to leave off this list?
Bond 007 - Brilliant multiplayer and the best FPS mission structure consoles had ever seen at that point.
BattleTanx - Another brilliant multiplayer FPS that is often overlooked.
The New Tetris - Graphics, sound, and controls were so smooth. Different play modes, plenty of unlockable targets, and top-notch multiplayer. Arguably the best console Tetris game to date.
Star Wars Episode I Pod Racer - Incredibly fast racing, fun dual-analog control option, cool upgrade system.
Zelda: Majora's Mask - The living world and time-based gameplay is still stunning; even moreso as a departure from the usual Zelda formula.