If you're in an area that gets hot enough in summer for you to want cooling, then yes you'll be pumping heat into the ground during summer and extracting it in the winter. For those of us who don't get hot enough in summer to want cooling, we're always extracting heat from the ground, and so the ground we're extracting heat from gets cold, which reduces the efficiency of the system. Having a solar powered mechanism for putting some heat back into the ground in summer would help alleviate this problem.
Another approach would be to have a longer length of piping. The rate at which the soil temperature drops when you extract heat, and then 'recharges' naturally, will vary greatly if you have 100m vs 1000m of piping. This is why ground source heat pumps use pipes laid out in coils, as it greatly increases the surface area.
Apartment buildings in Finland have these 100m to 200m drill holes for the heat loops since you can't run those horizontally to the neighbor's yard. A bonus benefit is that the core of Earth has to basically cool down before they run out of heat to extract.
