Armstrong, involved in a fight with Frenchman Gilles Bouvard on Thursday's fifth stage, abandoned complaining of sickness.
The 24-year-old American, twice a stage winner, was riding in his final Tour for Motorola, who are withdrawing their sponsorship at the end of the season.
Eddy Seigneur, the 1995 French champion, also called it a day after 60km -- his second successive abandonment from the Tour.
The 27-year-old Seigneur, a teammate of yellow jersey leader Stephane Heulot, abandoned along with compatriot Cyril Robin, ninth in the Tour of Italy.
The peloton, cycling in very difficult weather conditions, also lost Italian Davide Bramati, the third member of the Panaria team to abandon since Saturday, and another Frenchman Frederic Pontier, teammate of Cyril Saugrain winner of Wednesday's fourth stage.
Earlier Italian Gian Matteo Fagnini, teammate of sprint king Mario Cipollini, had not started the stage because of a broken collarbone and Alexander Gonchenko of Russia had also failed to start.
Their retirements left 176 of the original 198 starters still in the Tour.
The 23-year-old Austrian, riding in his first Tour, is dismissive of the little Italian, twice a top three finisher in the Tour. "Chiappucci is not the key man in our team. He is pretty ordinary really and he does nothing except for himself -- he has never helped me," Luttemberger said.
Luttemberger suffered a blow when the man he pinpointed as Carrera's leader, Enrico Zaina, second in the Tour of Italy, abandoned on Tuesday after failing to recover from a fall the day before. "I am here to learn and to perform well in the mountains. There is no pressure on me and I am free to do what I like," he said.
Carrera team manager Davide Boifava gave a rather different interpretation of the Austrian's role. "Chiappucci is the team leader. Luttemberger is young and inexperienced and his role is to do nothing else but help Chiappucci," he said.
Carrera's problems will need to be sorted out before the mountain stages start in earnest on Saturday. The team already had to start the Tour without climbing star Marco Pantani who is recovering from a broken leg.
Luttemberger, educated at an English school near Graz, is a rarity among Austrian sportsmen in that he is a specialist in going up mountains whereas most prefer to ski down them in search of sporting glory.
He had a disastrous start to 1996 breaking both his arms in a fall in the Tour of Sardinia. But he came back with typical Austrian determination to finish third in May on the final stage of the Vasca race. Ahead of him were Miguel Indurain, five-time winner of the Tour de France, and Switzerland's Alex Zulle, second in the 1995 Tour.
The Austrian, lying 46th overall five minutes behind race leader Stephane Heulot, is bullish about his prospects of winning a mountain stage.
"I realise that a stage victory for me would be a miracle. But I know I am among the best climbers in the world and I am confident that I can win one," Luttemberger said.
But he has also to prevail over Chiappucci and it's not for nothing the Italian is called the 'little devil.'