Nonlinear and ultrafast optical (far infrared to near ultraviolet) spectroscopic techniques are versatile tools for probing material dynamics and symmetries. Recent advances in the design of X-ray free electron lasers now allow for the extension of ultrafast spectroscopy to the X-ray region. We discuss the examples of such extensions highlighting the unique contributions that implementation of ultrafast and nonlinear spectroscopy in the X-ray region can make to deepen our understandings of chemical and physical phenomena and material properties.