Multiplexing or muxing, when speaking of video and video editing, means basically a process where separate parts of the video (or 'streams' as they're called in video terminology) are joined together into one file.
Typical process of multiplexing is where we have MPEG-2 video file (extension normally .m2v), an AC3 audio file and a subtitle file and we want to combine into MPEG-2 system stream file in order to burn that file to SuperVideoCD disc. The same goes for DVD-video and more recently, AVCHD.
Opposite process is called as demultiplexing.
The actual process of muxing, technically, is the interleaving of an audio and video stream into one transport stream.
A demuxing software, can extract an audio or video stream from a complete transport stream in a container which then allows for separate processing. Audio may need to be converted and demuxing allows you to have the audio stream by itself which can then replace the old audio stream afterwards.