To make the churros, in a medium saucepan melt butter over medium heat. Stir in water, sugar, and salt. Bring to a boil. Reduce heat and stir in flour. Continue to stir over low heat for about 1 minute. Remove from heat and allow to cool. Note, make sure the dough has cooled in order to prevent eggs from scrambling. Incorporate 1 egg at a time into the dough, stirring until well mixed and the dough is smooth.
Prepare a pastry bag fitted with a large star tip. Set aside. Prepare cinnamon sugar by mixing together granulated sugar and ground cinnamon in a shallow pan. Line baking sheets with paper towels and set aside near frying station. Using a wok, or large saucepan, heat vegetable oil to a temperature of 350°F. Pipe 4-inches of dough into the oil, no more than three at a time. Fry until golden brown, using tongs to turn the churros. Note, maintaining the oil temperature at around 350°F is important. If the oil becomes too hot, the churros will brown quickly on the outside, not thoroughly cooking through. Remove crispy churros from the oil and roll into the cinnamon sugar. Set on prepared baking sheets to cool. Continue steps until all the dough has been fried.
Allow churros to cool completely. Slice horizontally in half. Line three halves of the churros side by side. Fill with ice cream. Top with three more churro halves. Wrap in plastic wrap and freeze until firm, at least 2 hours.
While sandwiches are freezing, place chocolate, whipping cream, and chili powder (optional) in a bain marie. Melt chocolate over low heat. Continue to stir until smooth. Allow to cool to room temperature. Dip frozen sandwiches into chocolate. Serve immediately or store in freezer.