You may also want to look at Arizona State, UNC, USC, and UCLA - big quant programs (although of course they should match up to your interests). Fordham has a good psychometrics/measurement & evaluation program, too.
By Columbia, do you mean their Teachers College PhD program? because the psychology department at Columbia GSAS (where I studied) doesn’t have quantitative psychology. You may want to check out TC’s funding for that particular PhD program, because not all PhD programs at TC are funded the same way that they are in GSAS.
Anyway, many quant psych people come in with less math/stats than you’d expect - because it’s such a little-known area that a lot of people switch to it later. I’m assuming you already took calculus as part of your pre-med requirements; some quant psych programs may want you to finish the calculus sequence, but may allow you to do that as a PhD student (one of my friends did this at UNC). You may have only taken three psychology classes in the psych department, but I bet that a lot of your neuroscience classes have significant psychology concepts/content in them - so I would think about that and see how your neuroscience coursework might satisfy a lot of the recommended classes for a psychology department.
The short version is that I do think it’s worth applying - especially if your quant psych research interests are in neuroscience. You should plan to explain how your neuroscience coursework is relevant to the foundational psych knowledge you need. You could also plan your Plan B to be to take some additional psych classes part-time next year in case you don’t get in anywhere - you’d only need a year and maybe 2-3 additional classes, if any.
**
Also, extra point. Which degree you get is going to, in a lot of ways, determine the kind of thing you can do post-college. if your goal is industry don’t worry about this so much - EdD vs. PhD and which department you study in doesn’t necessarily matter as much, although the education department’s M&E might be better for working at test prep companies and educational software companies. But industry is more focused on what you can do.
However, if you want to be a professor, where you get your PhD is going to really have an effect on where you can work. If you get a PhD in quantitative psychology, you could work in a psychology department, in an interdisciplinary quantitative methods program, probably in a school of education in an M&E department (if your research is relevant), even potentially a department of statistics depending on your level of expertise (I’ve seen this happen). If you get an EdD, the latter three options are more or less open to you, but getting a position in a psychology department will be difficult.