Scarlett Johansson was pregnant during filming, so many of her scenes were scheduled early in filming before she began to show. To help hide her pregnancy later in filming, three stunt doubles were hired. This caused a lot of confusion amongst the other actors since, according to them, all of the stunt women looked very similar to Johansson. Chris Evans stated that it got to the point where he would say hello and start a conversation with one of them only to realize midway that the person he was talking to wasn't Johansson. Ultimately, some scenes used CGI to hide Johansson's belly.
Because Ultron was eight to nine feet tall, the five-foot ten-inch James Spader had to wear an antenna-like contraption made out of a thick piece of wire, with two red balls attached to the top that went up his entire back and three feet above his head. This was done so that the cast members that shared scenes with him would have a reference point for where his eyes would be. The two red balls represented the placement of Ultron's eyes. Elizabeth Olsen stated that this was actually distracting, because Spader would be giving an intense performance and out of instinct, she would look at him rather than the balls representing his eyes. Much to everyone's amusement, whenever this happened, Aaron Taylor-Johnson would yell "Red balls! Look at his balls, Lizzie!" at her in order to get her to look in the right direction.
Aaron Taylor-Johnson was asked to lose weight (muscle mass in particular) because he looked too strong to portray a runner, who is usually trim.
James Spader was Joss Whedon's only choice for the role of Ultron, because of his "hypnotic voice that can be eerily calm and compelling, while also being very human and humorous."
Thanos was considered to be the next villain, but Joss Whedon decided against him in favor of Ultron: "We have to stay grounded. It's part of what makes the Marvel Universe click, their relationship to the real world. Thanos is not out of the mix, but Thanos was never meant to be the next villain. He's always been the overlord of villainy and darkness." Instead, Thanos would appear in Avengers: Infinity War (2018) and Avengers: Endgame (2019).