yboutros

When training a transformer on positionally encoded embeddings, should the tgt output embeddings also be positionally encoded? If so, wouldn't the predicted/decoded embeddings also be positionally encoded?

Went through the pain of packaging a python project on Nixos. Here's some issues I hit, and how I got lucky resolving them. I feel the most reliable way of doing this in the future is to use docker and just imperatively build.

Here's how I got web drivers, AI dependencies, gpu dependencies, and an api dependency bundled together into an ephemeral shell for python development, on NixOS 23.11

0. Enable Flakes

1. Start with setting up poetry2nix

2. Get the template flake by running nix flake init --template github:nix-community/poetry2nix

3. in the flake.nix, sometimes changing projectDir = self to projectDir = ./. fixed some issues

4. in your terminal, run nix develop . to build the poetry app with python packages described in pyproject.toml

5. By default, just poetry and python latest should be installed. the dependencies for the project (which gets reflected in the pyproject.toml) are updated with poetry add, such as poetry add numpy selenium scikit-learn

6. Exit out of the ephemeral shell from nix develop ., and rerun to have poetry2nix rebuild and link the newly declared packages

Poetry2nix has worked pretty well for the more obscure python packages, but failed in others. For example, sentence-transformers would depend on maturin, which would fail to link setuptools. If poetry doesn't work, you can try and get the package from nixpkgs, or specify sha256s from pypi.org

Here's an example of what I added to my flake.nix to get gpu acceleration, sentence-transfomers, firefox drivers for selenium, and other packages poetry failed to setup:

packages = [ pkgs.poetry pkgs.python311Packages.sentence-transformers pkgs.firefox 
            pkgs.python311Packages.openai pkgs.python311Packages.yt-dlp pkgs.python311Packages.pyopencl
];

was added to this flake.nix, as in,

{
  description = "Application packaged using poetry2nix";

  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    poetry2nix = {
      url = "github:nix-community/poetry2nix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
  outputs = { self, nixpkgs, flake-utils, poetry2nix }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        # see https://github.com/nix-community/poetry2nix/tree/master#api for more functions and examples.
        pkgs = nixpkgs.legacyPackages.${system};
        inherit (poetry2nix.lib.mkPoetry2Nix { inherit pkgs; }) mkPoetryApplication;
      in
      {
        packages = {
          myapp = mkPoetryApplication {
            projectDir = ./.;
          };
          default = self.packages.${system}.myapp;
        };
        devShells.default = pkgs.mkShell {
          inputsFrom = [ self.packages.${system}.myapp ];
          packages = [ pkgs.poetry pkgs.python311Packages.sentence-transformers pkgs.firefox 
            pkgs.python311Packages.openai pkgs.python311Packages.yt-dlp pkgs.python311Packages.pyopencl
          ];
          nativeBuildInputs = [(
            pkgs.python311Packages.buildPythonPackage rec {
              pname = "serpapi";
              version = "0.1.5";
              src = pkgs.python311Packages.fetchPypi {
                inherit pname version;
                sha256 = "b9707ed54750fdd2f62dc3a17c6a3fb7fa421dc37902fd65b2263c0ac765a1a5";
              };
            }
          )];
        };
      });
}

There was one package (serpapi), which was not in nixpkgs, and poetry failed as well. Adding this to native build inputs got serpapi installed

nativeBuildInputs = [(
            pkgs.python311Packages.buildPythonPackage rec {
              pname = "serpapi";
              version = "0.1.5";
              src = pkgs.python311Packages.fetchPypi {
                inherit pname version;
                sha256 = "b9707ed54750fdd2f62dc3a17c6a3fb7fa421dc37902fd65b2263c0ac765a1a5";
              };
            }
)];

All in all, it works, and I have no doubt I've made a reproducible environment. What attracts me is I've never had an easier time setting up cuda/cudnn/tensorrt/... system drivers have been near effortless, and much faster to setup than on debian. Tools like sentence-transformers and torch default to packages which leverage the GPU.

What pushes me away, is I've had failures in each of the three methods for specifying package dependencies, even though one of the three eventually was the fix for integrating the dependencies into my shell. For now, I'll stick with it, but it's hard for me to suggest to a team we use this in development

I setup a next.js project with pkgs.mkshell, and used nix develop to automatically build the project. However, when I leave the shell, the files persist. How should/can(?) I setup my shell.nix so that files in the directory it drops down into are automatically removed when leaving the ephemeral shell?