ICP was founded as a label and group concept in the late '60s when reedist Willem Breuker and drummer Han Bennink self-released a duo recording on a label they named ICP. Fellow Dutch musician and collaborator, pianist Misha Mengelberg became the third equal member of ICP, having recently come up with the term "instant composing" to use in "improvising's" stead. The name ICP quickly became an umbrella for a wide variety of line-ups during its first decade of existence. It wasn't too long before these groups included either Breuker or Mengelberg but not both, as the two musicians had clashing opinions on approaches to live performance, what ICP should be and many other musical issues. Breuker wanted tunes and rehearsals, Mengelberg wanted instant composing. Breuker wanted more people admitted who would have equal voting rights, Mengelberg wanted the core three members to have final say. Both also had different takes on music theater, which ICP got involved in the late '60s. So, both led their own ICP gigs, with Bennink (who didn't choose sides) performing in both, although more often with Mengelberg. Breuker's group included such musicians as bassist Maarten Altena, trombonist Willem van Manen, saxophonist Peter Bennink (Han's younger brother), pianist Leo Cuypers and keyboardist Michel Waisvisz. Mengelberg and Bennink first had a trio with the frequently-visiting British saxophonist Evan Parker starting in 1969, followed by a quartet line-up with reedsman John Tchicai and guitarist Derek Bailey, which had a brief tour and two recordings from 1970-71, including a classic of European improvisation, Fragments. Months before this, the core duo and Parker were joined by Bailey, saxophonist Peter Brotzmann, trombonist Paul Rutherford and Peter Bennink to record a septet album, Groupcomposing. During this time, Mengelberg and Bennink also recorded an untitled duo for the ICP label.

By 1973, the inevitable split came and musicians went with one or the other, resulting in Breuker's still-thriving theatrical and fun-loving Willem Breuker Kollektief, while Mengelberg continued with the name ICP, which was, again, a continuously changing line-up for several years, during which only Misha, Han Bennink and American tubist Larry Fishkind were mainstays. Rotating members included Brotzmann, Tchicai, cellist Tristan Honsinger, saxophonist Keshavan Maslak, trombonist George Lewis and saxophonist Paul Termos. The first recording of ICP in its later, larger size came in early 1977, as ICP-Tentet in Berlin (SAJ, 1978). The first of the lasting members came three years later, when trombonist Wolter Wierbos joined. Reedsman Michael Moore came on board not too long after, first appearing on Japan Japon (DIW, 1982), as did violist Maurice Horsthuis, who was soon followed by his Amsterdam String Trio bandmates, bassist Ernst Glerum and cellist Ernst Reijseger (Reijseger, Moore and Bennink also formed another group together, the Clusone 3). Horsthuis eventually left (and the violist chair was filled a couple more times before being retired) but Reijseger and Glerum stayed. Due to the larger roster, the "instant composing" tenet shifted slightly to "conducted improvisation." By the 1990s, the band also included saxophonist Ab Baars and trumpeter Thomas Heberer, with Honsinger returning by the mid-'90s.

ICP Orchestra has recorded tributes to Thelonious Monk and Herbie Nichols (ICP 026) in 1984 and 1986, respectively. Mengelberg has also led the group in many brilliant (but unrecorded) programs of Duke Ellington's music. In addition to the Monk/Nichols release, and the early '80s DIW Japan Japon, ICP also recorded two volumes of Bodspaadje Konijnehol (Forest Path Rabbithole) from 1989-90 (ICP 028,029). The ICP Orchestra didn't record again until 1997, this time for HatArt, resulting in the critically hailed album Jubilee Varia. The line-up by then consisted of Mengelberg, Moore, Baars, Wierbos, Reijseger, Honsinger, Glerum and Bennink, but by the time the group toured for the album's 1999 release, Reijseger had left the group permanently.