Mostly the fact that GrapheneOS only works on Google Pixel hardware currently and vendor unlock status. It's the only available phone hardware that provides full bootloader unlock capabilities AND suitable security protections baked into the secure enclave and boot process, including things like rate limiting in hardware like password cracking attempts via external brute-force input means, lockdown of usb ports until boot unlocked with a pin, etc. Their website spells out all the reasons.

Other phone makers could if they wanted to do the same, but do not as an active choice, or at least somebody's choice above them.

Custom ROMs fail device integrity, which means you cannot use banking, financial, government, payments and telcom apps, not to mention all the games that refuse to work.

... secure boot?

I don't understand "just load GrapheneOS" sentiments. It only runs on extremely specific flagship devices with explicit features that allow it that are out of financial and technical reach for >99.9% of population of Earth and it still fully relies on AOSP. It's an escape hatch for mice. Or is it really not that way?

It will be used as evidence that the person who has GrapheneOS on their phone is attempting to break the law. Telegram and Signal chats are often used as circumstantial evidence of malfeasance in Indian national security cases, so the jump to using GrapheneOS as evidence of malfesance is tiny.