The basic VFR weather minimums (14 CFR 91.155) are specific to types of airspace and altitudes. Understanding the rationale behind the different requirements might help you remember them more easily.
VFR flight is based on the principle of “see and avoid.” The presumption made in establishing the basic VFR weather minimums is that aircraft flying at lower altitudes (i.e., below 10,000 MSL) and/or in airspace with radar approach control and/or an operating control tower (i.e., Class B, C, and D airspace) will be moving more slowly, or that they will be under positive control. Consequently, these aircraft do not need as much flight visibility or as much distance from clouds to see and avoid other traffic.
Aircraft operating at higher altitudes (i.e., Class E airspace above 10,000 MSL) are likely to be not only faster, but also operating on instrument flight plans. The rationale for greater visibility and more distance from clouds when flying above 10,000 MSL is to give VFR pilots more time to see and avoid faster aircraft that are popping in and out of clouds.