Technically, an ATI Radeon 9800 as that was my first custom built computer in 2003. However, the ATI Rage IIc was the gpu inside my first desktop computer, an iMac G3 in 1998. But the first one I used was the VGC 12-bpp palette graphics of the Apple IIgs, where I was first introduced to computer games and upgrading the accelerator cards and memory to play new games with more demanding requirements in 1994.
If you don’t need to host but can run locally, GPT4ALL is nice, has several models to download and plug and play with different purposes and descriptions, and doesn’t require a GPU.