There isn't really a reason why the server couldn't be launched automatically from Emacs. This requires figuring out a few things though:

• Locate a suitable Python interpreter.
• Locate the Python script relatively to the package file.
• Query the user for connection options if necessary (unless they set them explicitly).
• Pass those options to the Python script and launch it asynchronously.

The design for this can be lifted from other interaction packages, such as CIDER or Sly. See also #3.

Another thing worth thinking about is whether this really needs to use a HTTP service. If this is designed to run on localhost, talking with a subprocess makes a lot more sense and avoids security issues. I wrote a bunch before, though for interaction via Scheme rather than Emacs Lisp.

It's customary to provide a requirements.txt file to install Python dependencies. From what I can tell this would be Flask and https://huggingface.co/transformers/installation.html.

Hi, I am at the last stage of the setup and emacs just gets stuck on Contacting localhost:9900 and the terminal outputs Setting pad_token_id to eos_token_id:50256 for open-end generation but nothing happens. What am I meant to do?

device is set to "cuda" and changing it to "cpu" returns the same error

python server.py

Traceback (most recent call last):
  File "A:\xxxxxxxxx\emacs-secondmate\serve\server.py", line 10, in <module>
    model = AutoModelForCausalLM.from_pretrained(modelname)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\auto\auto_factory.py", line 395, in from_pretrained
    return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\modeling_utils.py", line 1179, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 905, in __init__
    self.transformer = GPTNeoModel(config)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 708, in __init__
    self.h = nn.ModuleList([GPTNeoBlock(config, layer_id=i) for i in range(config.num_layers)])
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 708, in <listcomp>
    self.h = nn.ModuleList([GPTNeoBlock(config, layer_id=i) for i in range(config.num_layers)])
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 542, in __init__
    self.mlp = GPTNeoMLP(inner_dim, config)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 521, in __init__
    self.c_fc = nn.Linear(embed_dim, intermediate_size)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\torch\nn\modules\linear.py", line 81, in __init__
    self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))
RuntimeError: [enforce fail at ..\c10\core\CPUAllocator.cpp:79] data. DefaultCPUAllocator: not enough memory: you tried to allocate 104857600 bytes.

If the user has access to the HuggingFace Accelerated Inference API [0], allow them to configure appropriate API Tokens and inference the model via the API rather than having to run the Language Model server locally.

[0] https://api-inference.huggingface.co/docs/python/html/index.html

• Remove top-level key binding commands and instead document what commands users should bind in their init file.
• Add package header and footer.
• Make sure that the package adheres to MELPA's coding guidelines.

See also #3.

Hi, I love the project and would like to try it out. However, as an emacs noob I'm not sure how to get this started with something like use-package.

Additionally, does the plugin automatically launch the server, or would you have to do that manually? If so, could it be made to be automatic?