I think no is the short answer. You can't edit Quantlib libraries in Quantlib Python - you can only use them.
Here's my understanding: Quantlib is a library written in C++ with all source files freely available. Once compiled the library can be "accessed" from a scripting language like Python, R ... etc via specially written interface files through a program called SWIG. Not all Quantlib C++ classes are accessible via SWIG - only ones for which an interface file has been created. So using Quantlib Python is just a front end that lets you utilize Quantlib C++. To do what you're suggesting would involve:
- Download the Quantlib C++ source package and compile/install on a local workstation
- Download and install Quantlib SWIG which creates a Python module that interfaces with that particular C++ installation
- Once you have everything working, you can change the source for any C++ file to your own customization, recompile the code and reinstall.
- To get these changes working with Python: depending on what you've changed/added in C++ you may need to either edit an existing/create a new interface file, wrap them and reinstall Quantlib-SWIG.
I don't know the details of your particular issue: but I think you would need to create a new class called MonotoneConvex and tell the FittingMethod interface file about this new class. Alternatively, you can raise a community GitHub request to add this new method to the master library (or, since Quantlib is community driven project and you know what you're doing, contribute it yourself!)
Btw if, like myself, you sometimes just want to get things to work quickly so you can play around with ideas and you're not a C++ whiz-kid: you can do steps 1,2 and 3 above and then just hack the C++ code of one of the existing method classes to the math you want executed. Then recompile the C++ library (this way you don't even have to change the SWIG installation). And things will run in your local Python module. This may work depending on what dependencies your hack breaks...etc (you'll find this out when you try and recompile the altered C++ code).