### Bug Description I have a rock (rockcraft bellow) with one service which I need to deploy in Pod in Kubernetes cluster. The Pod's container spec looks like this (the Pod is created by upstream Kubernetes controller): ``` ... containers: - args: - --model_name=pmml-demo - --model_dir=/mnt/models - --http_port=8080 image: misohu/pmmlserver:v0.10.0_22.04_1 imagePullPolicy: IfNotPresent ... ``` We need to find a way for those args in Pod manifest to end up in Rock's service (without adjusting the Pod's manifest, because it is created by an upstream Kubernetes controller). If I understand correctly by running the Pod like this I will just send the args to the pebble not to the service (based on [this](https://github.com/canonical/pebble#running-the-daemon-server)). I know that if I want to pass args to the specific service I can do it like this: ``` docker run misohu/pmmlserver:v0.10.0_22.04_1 --args pmml-server --model_name=pmml-demo ``` But is there a way for us (kubeflow team) to pass the args to the one and only service which we will run in the rock without adjusting the Pod's manifest please? This is a huge problem for us because we cannot create Rocks for models servers whose Pods get dynamically created from upstream controllers. cc: @kimwnasptd ### To Reproduce I built the rock with ``` rockcraft clean && rockcraft pack --verbosity=trace sudo skopeo --insecure-policy copy oci-archive:pmml-server_v0.10.0_22.04_1_amd64.rock docker-daemon:misohu/pmmlserver:v0.10.0_22.04_1 docker push misohu/pmmlserver:v0.10.0_22.04_1 ``` Then we deploy pod (its upstream manifest of Pod which we cannot adjust) ``` apiVersion: v1 kind: Pod metadata: annotations: internal.serving.kserve.io/storage-initializer-sourceuri: gs://kfserving-examples/models/pmml serving.kserve.io/deploymentMode: RawDeployment serving.kserve.io/enable-metric-aggregation: "false" serving.kserve.io/enable-prometheus-scraping: "false" labels: app: isvc.pmml-demo-predictor component: predictor pod-template-hash: 77d5674687 serving.kserve.io/inferenceservice: pmml-demo name: pmml-demo-predictor-77d5674687-jxq4j namespace: kubeflow spec: containers: - args: - --model_name=pmml-demo - --model_dir=/mnt/models - --http_port=8080 image: misohu/pmmlserver:v0.10.0_22.04_1 imagePullPolicy: IfNotPresent name: kserve-container readinessProbe: failureThreshold: 3 periodSeconds: 10 successThreshold: 1 tcpSocket: port: 8080 timeoutSeconds: 1 resources: limits: cpu: "1" memory: 500Mi requests: cpu: 100m memory: 250Mi terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-lxfj6 readOnly: true - mountPath: /mnt/models name: kserve-provision-location readOnly: true dnsPolicy: ClusterFirst enableServiceLinks: true initContainers: - args: - gs://kfserving-examples/models/pmml - /mnt/models image: kserve/storage-initializer:v0.11.0 imagePullPolicy: IfNotPresent name: storage-initializer resources: limits: cpu: "1" memory: 1Gi requests: cpu: 100m memory: 100Mi terminationMessagePath: /dev/termination-log terminationMessagePolicy: FallbackToLogsOnError volumeMounts: - mountPath: /mnt/models name: kserve-provision-location - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-lxfj6 readOnly: true nodeName: hyrule preemptionPolicy: PreemptLowerPriority priority: 0 restartPolicy: Always schedulerName: default-scheduler securityContext: {} serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations: - effect: NoExecute key: node.kubernetes.io/not-ready operator: Exists tolerationSeconds: 300 - effect: NoExecute key: node.kubernetes.io/unreachable operator: Exists tolerationSeconds: 300 volumes: - name: kube-api-access-lxfj6 projected: defaultMode: 420 sources: - serviceAccountToken: expirationSeconds: 3607 path: token - configMap: items: - key: ca.crt path: ca.crt name: kube-root-ca.crt - downwardAPI: items: - fieldRef: apiVersion: v1 fieldPath: metadata.namespace path: namespace - emptyDir: {} name: kserve-provision-location ``` ### Environment Rockcraft is not failing. ### rockcraft.yaml ```shell # Based on https://github.com/kserve/kserve/blob/v0.10.0/python/pmml.Dockerfile name: pmml-server summary: Pmml server for Kserve deployments description: "Kserve Pmml server" version: "v0.10.0_22.04_1" license: Apache-2.0 base: ubuntu:22.04 build-base: ubuntu:22.04 platforms: amd64: run-user: _daemon_ services: pmml-server: override: replace summary: "Pmml server service" startup: enabled command: python3.10 -m pmmlserver environment: PYTHONPATH: /pmmlserver:/kserve parts: security-team-requirement: plugin: nil override-build: | mkdir -p ${CRAFT_PART_INSTALL}/usr/share/rocks (echo "# os-release" && cat /etc/os-release && echo "# dpkg-query" && \ dpkg-query --root=${CRAFT_PROJECT_DIR}/../bundles/ubuntu-22.04/rootfs/ -f '${db:Status-Abbrev},${binary:Package},${Version},${source:Package},${Source:Version}\n' -W) \ > ${CRAFT_PART_INSTALL}/usr/share/rocks/dpkg.query python: plugin: nil source: https://github.com/kserve/kserve.git source-subdir: python source-tag: v0.10.0 build-packages: - python3.10 overlay-packages: - openjdk-11-jdk - python3.10 - python3.10-distutils - python3-setuptools override-build: | cd python pip3 install --no-cache-dir --upgrade pip pip3 install -e ./kserve pip3 install --no-cache-dir -e ./pmmlserver cp -fr pmmlserver $CRAFT_PART_INSTALL/pmmlserver cp -fr kserve $CRAFT_PART_INSTALL/kserve mkdir -p $CRAFT_PART_INSTALL/usr/local/lib/python3.10/dist-packages cp -fr /usr/local/lib/python3.10/dist-packages/* $CRAFT_PART_INSTALL/usr/local/lib/python3.10/dist-packages/ third-party: plugin: nil after: [python] source: https://github.com/kserve/kserve.git source-subdir: python source-tag: v0.10.0 override-build: | cp -fr third_party/* ${CRAFT_PART_INSTALL}/third_party ``` ### Relevant log output ```shell Defaulted container "kserve-container" out of: kserve-container, storage-initializer (init) error: unknown flag `model_name' ```