source activate peft make transformers_tests shell: bash --noprofile --norc -e -o pipefail {0} env: RUN_SLOW: yes IS_GITHUB_CI: 1 NVIDIA_DISABLE_REQUIRE: 1 SLACK_API_TOKEN: *** CUDA_VISIBLE_DEVICES: 0 TEST_TYPE: single_gpu_huggingface/peft-gpu-bnb-source:latest RUN_SLOW=1 python -m pytest transformers-clone/tests/quantization/bnb --report-log "transformers_tests.log" ============================= test session starts ============================== platform linux -- Python 3.8.18, pytest-8.0.0, pluggy-1.4.0 rootdir: /__w/peft/peft/transformers-clone configfile: pyproject.toml plugins: anyio-4.2.0, reportlog-0.4.0, cov-4.1.0, xdist-3.5.0 collected 82 items transformers-clone/tests/quantization/bnb/test_4bit.py ..............s.. [ 20%] .........FFFFFFFFFFF.. [ 47%] transformers-clone/tests/quantization/bnb/test_mixed_int8.py .........F. [ 60%] ......F...sssss..........F...... [100%] =================================== FAILURES =================================== ___________________ BaseSerializationTest.test_serialization ___________________ self = quant_type = 'nf4', double_quant = True, safe_serialization = True def test_serialization(self, quant_type="nf4", double_quant=True, safe_serialization=True): r""" Test whether it is possible to serialize a model in 4-bit. Uses most typical params as default. See ExtendedSerializationTest class for more params combinations. """ tokenizer = AutoTokenizer.from_pretrained(self.model_name) self.quantization_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type=quant_type, bnb_4bit_use_double_quant=double_quant, bnb_4bit_compute_dtype=torch.bfloat16, ) model_0 = AutoModelForCausalLM.from_pretrained( self.model_name, quantization_config=self.quantization_config, device_map=torch_device, ) size mismatch for transformer.h.33.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.33.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.34.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.34.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.34.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.35.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.35.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.35.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.36.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.36.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.36.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.37.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.37.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.37.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.38.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.38.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.38.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.39.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.39.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.39.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.40.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.40.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.40.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.41.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.41.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.41.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.42.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.42.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.42.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.43.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.43.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.43.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.44.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.44.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.44.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.45.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.45.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.45.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.46.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.46.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.46.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). size mismatch for transformer.h.47.attn.c_attn.weight: copying a param with shape torch.Size([4800, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 4800]). size mismatch for transformer.h.47.mlp.c_fc.weight: copying a param with shape torch.Size([6400, 1600]) from checkpoint, the shape in current model is torch.Size([1600, 6400]). size mismatch for transformer.h.47.mlp.c_proj.weight: copying a param with shape torch.Size([1600, 6400]) from checkpoint, the shape in current model is torch.Size([6400, 1600]). You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method. ====== 14 failed, 62 passed, 6 skipped, 25 warnings in 547.97s (0:09:07) =======