TextToImage

• Original Link : https://keras.io/api/keras_hub/base_classes/text_to_image/
• Last Checked at : 2024-11-26

source

TextToImage class

keras_hub.models.TextToImage()

Base class for text-to-image tasks.

TextToImage tasks wrap a keras_hub.models.Backbone and a keras_hub.models.Preprocessor to create a model that can be used for generation and generative fine-tuning.

TextToImage tasks provide an additional, high-level generate() function which can be used to generate image by token with a string in, image out signature.

All TextToImage tasks include a from_preset() constructor which can be used to load a pre-trained config and weights.

Example

# Load a Stable Diffusion 3 backbone with pre-trained weights.
text_to_image = keras_hub.models.TextToImage.from_preset(
    "stable_diffusion_3_medium",
)
text_to_image.generate(
    "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
)
# Load a Stable Diffusion 3 backbone at bfloat16 precision.
text_to_image = keras_hub.models.TextToImage.from_preset(
    "stable_diffusion_3_medium",
    dtype="bfloat16",
)
text_to_image.generate(
    "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
)

source

from_preset method

TextToImage.from_preset(preset, load_weights=True, **kwargs)

Instantiate a keras_hub.models.Task from a model preset.

A preset is a directory of configs, weights and other file assets used to save and load a pre-trained model. The preset can be passed as one of:

1. a built-in preset identifier like 'bert_base_en'
2. a Kaggle Models handle like 'kaggle://user/bert/keras/bert_base_en'
3. a Hugging Face handle like 'hf://user/bert_base_en'
4. a path to a local preset directory like './bert_base_en'

For any Task subclass, you can run cls.presets.keys() to list all built-in presets available on the class.

This constructor can be called in one of two ways. Either from a task specific base class like keras_hub.models.CausalLM.from_preset(), or from a model class like keras_hub.models.BertTextClassifier.from_preset(). If calling from the a base class, the subclass of the returning object will be inferred from the config in the preset directory.

Arguments

• preset: string. A built-in preset identifier, a Kaggle Models handle, a Hugging Face handle, or a path to a local directory.
• load_weights: bool. If True, saved weights will be loaded into the model architecture. If False, all weights will be randomly initialized.

Examples

# Load a Gemma generative task.
causal_lm = keras_hub.models.CausalLM.from_preset(
    "gemma_2b_en",
)
# Load a Bert classification task.
model = keras_hub.models.TextClassifier.from_preset(
    "bert_base_en",
    num_classes=2,
)

source

compile method

TextToImage.compile(optimizer="auto", loss="auto", metrics="auto", **kwargs)

Configures the TextToImage task for training.

The TextToImage task extends the default compilation signature of keras.Model.compile with defaults for optimizer, loss, and metrics. To override these defaults, pass any value to these arguments during compilation.

Arguments

• optimizer: "auto", an optimizer name, or a keras.Optimizer instance. Defaults to "auto", which uses the default optimizer for the given model and task. See keras.Model.compile and keras.optimizers for more info on possible optimizer values.
• loss: "auto", a loss name, or a keras.losses.Loss instance. Defaults to "auto", where a keras.losses.MeanSquaredError loss will be applied. See keras.Model.compile and keras.losses for more info on possible loss values.
• metrics: "auto", or a list of metrics to be evaluated by the model during training and testing. Defaults to "auto", where a keras.metrics.MeanSquaredError will be applied to track the loss of the model during training. See keras.Model.compile and keras.metrics for more info on possible metrics values.
• **kwargs: See keras.Model.compile for a full list of arguments supported by the compile method.

source

save_to_preset method

TextToImage.save_to_preset(preset_dir)

Save task to a preset directory.

Arguments

• preset_dir: The path to the local model preset directory.

preprocessor property

keras_hub.models.TextToImage.preprocessor

A keras_hub.models.Preprocessor layer used to preprocess input.

backbone property

keras_hub.models.TextToImage.backbone

A keras_hub.models.Backbone model with the core architecture.

source

generate method

TextToImage.generate(inputs, num_steps, guidance_scale, seed=None)

Generate image based on the provided inputs.

Typically, inputs contains a text description (known as a prompt) used to guide the image generation.

Some models support a negative_prompts key, which helps steer the model away from generating certain styles and elements. To enable this, pass prompts and negative_prompts as a dict:

text_to_image.generate(
    {
        "prompts": "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k",
        "negative_prompts": "green color",
    }
)

If inputs are a tf.data.Dataset, outputs will be generated “batch-by-batch” and concatenated. Otherwise, all inputs will be processed as batches.

Arguments

• inputs: python data, tensor data, or a tf.data.Dataset. The format must be one of the following:
  • A single string
  • A list of strings
  • A dict with “prompts” and/or “negative_prompts” keys
  • A tf.data.Dataset with “prompts” and/or “negative_prompts” keys
• num_steps: int. The number of diffusion steps to take.
• guidance_scale: float. The classifier free guidance scale defined in Classifier-Free Diffusion Guidance. A higher scale encourages generating images more closely related to the prompts, typically at the cost of lower image quality.
• seed: optional int. Used as a random seed.