I've partially covered this in another answer, which might be worth a read.
Libre is from the French and/or Spanish (there may be other languages too, but those come to mind). In said languages, "libre" implies, as you say, free speech not free beer.
The first difference, then, is that FLOSS is actually implied to be a bit more restrictive to the creator: i.e. categorising your software as FLOSS implies you will not charge for it. FOSS, on the other hand, doesn't specify what "free" it means, so you can charge for it.
Your users, on the other hand, won't see much of a difference apart from cost: whether software is FLOSS or FOSS, they will be able to use it in much the same way. This is as opposed to sourcing differences: the difference between open- and visible-source is large, whereas the difference between FOSS and FLOSS is small.
It seems this is the only major distinction between the two.
Stallman has also written this article, which mentions FOSS and FLOSS briefly, among other discussion of the principles of "free" software. (Thanks to Pandya for bringing this up in chat).