I want to double major in both and wanted to know how difficult it really is
See the following link:
https://www.advising.ufl.edu/academicinfo/duals-and-doubles/#notpermitted
Physics and Math are both CLAS (College of Liberal Arts and Sciences) majors and are under-enrolled (which makes it easier to get approval to do the double major or dual degree).
You have the choice of doing a double major or a dual degree.
A “Double Major Within a College” application (if both majors are in CLAS and both confer the same degree, i.e., both are B.A. degrees), or
A “Dual Degree Application” (used when the majors confer two different CLAS degrees, such as a BA in Philosophy and a BS in Geology, or when the majors are offered by two different colleges).
You can earn a BS or a BA in Math (note that the BA track is the most “efficient” way to get a dual degree with Physics):
The BS track is a selection of courses which will best prepare you for graduate school in Mathematics (as opposed to graduate school in Education, Business or Engineering.) Graduate school in Mathematics is an intense proof-oriented curriculum and the BS track narrows the selection of courses to focus on that target.
The BA track offers the greatest flexibility in the choice of electives and their availability. The BA allows the student to choose up to three courses from the elective list in areas like Computer Science, Industrial Engineering, Physics, and Statistics. Using the guidelines in Course Recommendations you can build a secondary specialty within the BA track. The BA is also the most efficient way for a student to get a dual degree in mathematics and one of these majors.
http://www.math.ufl.edu/mathematics-major/ba-and-the-bs-degrees/
You can also earn a BS or BA in Math:
http://www.phys.ufl.edu/academics/undergraduate/
Since both majors are “under-enrolled”, it’s even possible to do a triple major/degree. What fun!