The San Gabriel Valley was a great place for little hole-in-the-wall Chinese/Taiwanese variety stores and CD/VCD/DVD/book stores in the early 2000s, and even now, I think there are a reasonable number of such places left. I got very good at finding places that were likely to have slightly older CDs and VCDs at a discount, and when the price got down to about $2.00 each, I'd buy just about anything that looked interesting (which is how I got into Huang Fei, Tarcy, Cindy Chen, and Kelly Chen - and how I picked up a lot of good China Dolls and Yuki Hsu stuff as well).
Around that time, the SGV also had two very good Life Plaza stores. Life Plaza was a TW-based chain of variety store/cafes which carried lots of CDS/VCDs, aisles of character-based Taiwanese/Japanese/Korean stationery items, and random things like Hello Kitty steering wheel covers. The stores are long gone now (I'm not even sure that the chain still exists in Taiwan), but they were great while they lasted.
Eventually, I found the Book-Off stores in Gardena and Costa Mesa, and bought stacks of used Jpop CDs, DVDs, and VHS tapes (which was a great way to find out about somewhat older artists like Kudo Shizuka/Ushirogami Hikaretai, Tomosaka Rie, Princess Princess, Pink Sapphire, etc.).
Overall, I'd say that one of the best things about hole-in-the-wall/variety/used book/media stores is that they made it easy to discover older material which frequently didn't show up anywhere else back in the pre-Youtube/streaming era.