Games like Umihara Kawase BaZooKa!
Games like Umihara Kawase BaZooKa! include Cuphead, Hollow Knight: Silksong, Broforce, Celeste, and ICEY. We matched them by the tags players give Umihara Kawase BaZooKa! on Steam (Action, Female Protagonist, Competitive, 2D, and Difficult): closest matches first, and among equally close ones the better-known games.

Cuphead
Shared: Co-op · Difficult · 2D

Hollow Knight: Silksong
Shared: Difficult · Indie · 2D

Broforce
Shared: Action · Co-op · 2D

Celeste
Shared: Difficult · Female Protagonist · Indie

ICEY
Shared: Action · Female Protagonist · Indie

Skul: The Hero Slayer
Shared: Action · Indie · 2D

The Dishwasher: Vampire Smile
Shared: Action · Indie · 2D

Rift of the NecroDancer
Shared: Female Protagonist · Action · Difficult

The Vagrant
Shared: Action · Female Protagonist · 2D

Monster Hunter: World
Shared: Action · Co-op · Difficult

Just Shapes & Beats
Shared: Action · Difficult · 2D

Full Metal Furies
Shared: Action · Co-op · Indie

Nine Sols
Shared: Action · 2D · Difficult

ENDER LILIES: Quietus of the Knights
Shared: Female Protagonist · 2D · Difficult

Salt and Sanctuary
Shared: Action · 2D · Difficult

SpeedRunners
Shared: Competitive · Action · Indie

Hollow Knight
Shared: Difficult · 2D · Indie

Risk of Rain (2013)
Shared: Action · Indie · Difficult

Awesomenauts
Shared: Action · 2D · Co-op

Resident Evil 3
Shared: Action · Female Protagonist · Co-op

Barotrauma
Shared: Co-op · 2D · Action

Rubinite
Shared: Female Protagonist · Difficult · 2D

Minoria
Shared: Action · Female Protagonist · Indie

Bloodstained: Ritual of the Night
Shared: Female Protagonist · Action · Co-op