An ISP is an Internet Service Provider. Lots of organizations are ISPs.
The ISP is the business that you connect to that connects you to the internet. Some of the more commonly known ISPs are AT&T, Comcast, Ringgold Telephone, Northland Communications. Many of these are companies that originally were telephone or cable companies since they were already in the process of having the major communications connected to them and then sending signals to individual homes or businesses. Most often we still see the ISP running a physical wire (or fiber optic) cable to our house. We then connect to their network with either a router or modem (modems when using dialup)
You can think o the ISP as being similar to your gas station. For us to get gas to run our cars we go to the gas station to get it. However what the gas station is doing is being the delivery point to us of the oil from when drilled, transported, refined and piped and trucked to the gas station. The gas is out there before the gas station but not in a method to deliver to us until at station.
Likewise there are major cables run all over the USA and the world that are carrying Internet signals but to get them down to our house or business so we can connect out computer to the Internet signal requires an ISP.
In some places you are limited to one ISP and in many places now more than one ISP runs wiring to your location (or a satellite signal or wireless) that you can connect to.
Thanks to Clay for the question